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BACKGROUND 

The  present  research  is  the  second  in 
a  series  of  studies  to  develop  and  test 
a  Department  of  Defense  (DoD) 
Adjudication  Decision  Support  (ADS) 
system.  The  long-term  goal  is  to 
develop  an  ADS  system  that  will 
automatically  evaluate  completed 
personnel  security  investigations  in 
accordance  with  the  Adjudicative 
Guidelines  for  Determining  Eligibility 
for  Access  to  Classified  Information 
(December  29,  2005)  and  by  using 
decision  logic  comparable  to  expert 
adjudicators.  Cases  that  contain 
appropriately  little  adverse 
information  will  qualify  for  automatic 
clearance  approval.  The  purpose  of 
the  present  study  was  to  estimate 
automated  approval  rates  for  NACLC 
investigations  based  on  preliminary 
decision  rules. 


HIGHLIGHTS 

The  results  indicated  that 
approximately  40%  of  NACLC 
investigations  in  the  study  sample 
would  have  been  appropriate  for 
automated  approval  under  the 
reported  decision  rules  because  these 
cases  contained  little  or  no  adverse 
information  of  security  concern.  As 
part  of  the  Defense  Information 
Systems  for  Security  (DISS),  an  ADS 
system  could  provide  significant  cost 
savings,  improve  adjudication 
timeliness,  and  allow  the  central 
adjudication  facilities  to  focus  human 
resources  on  complex  cases  and  those 
with  serious  adverse  information. 
Machine-readable  data  from 
investigation  providers  will  be 
required  to  deploy  the  system. 
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PREFACE 

Automated  review  of  security  clearance  Reports  for  Adjudication  (RFA)  is  one  way 
the  Department  of  Defense  (DoD)  can  improve  the  efficiency  of  the  personnel 
security  program.  The  present  study  continues  the  development  of  decision  rules  to 
identify  National  Agency  Check  with  Local  Agency  Checks  and  Credit  Checks 
(NACLC)  investigations  with  adverse  information  that  does  not  pose  an  undue 
security  concern.  Decision  rules  were  tested  against  a  random  sample  of  DoD  RFAs. 
The  results  of  this  study  advance  our  understanding  of  the  relationships  between 
investigative  information  and  clearance  eligibility  determinations,  and  serve  as  the 
foundation  for  developing  an  automated  Adjudication  Decision  Support  (ADS) 
system. 


James  A.  Riedel 
Director 
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EXECUTIVE  SUMMARY 

The  present  research  is  the  second  in  a  series  of  studies  to  evaluate  preliminary 
decision  rules  and  provide  automated  approval  estimates  for  a  Department  of 
Defense  (DoD)  Adjudication  Decision  Support  (ADS)  system.  The  long-term  goal  of 
the  ADS  program  is  to  develop  a  system  that  will  automatically  evaluate  completed 
personnel  security  investigations  in  accordance  with  the  Adjudicative  Guidelines  for 
Determining  Eligibility  for  Access  to  Classified  Information  (December  29,  2005) 
and  using  decision  logic  comparable  to  the  logic  of  expert  adjudicators.  The  ADS 
system  will  process  closed  personnel  security  investigations  in  accordance  with 
national  guidelines  before  distribution  to  the  DoD  central  adjudication  facilities 
(CAFs).  The  system  will  identify  information  that  is  relevant  to  clearance  eligibility 
determinations  based  on  decision  rules  established  by  policy,  senior  adjudicators, 
other  personnel  security  experts,  and  empirical  research  on  past  determinations. 
Cases  that  contain  appropriately  little  adverse  information  will  qualify  for  automatic 
approval.  This  should  greatly  shorten  case  processing  times  for  the  cleanest  cases, 
facilitate  the  initial  review  of  complex  cases  by  adjudicators,  reduce  the  overall 
workload  at  the  CAFs,  increase  adjudication  consistency,  and  facilitate  case 
assignment. 

The  ADS  system  will  have  two  major  components:  (1)  an  automated  approval 
process  and  (2)  an  automated  case  summary  reporting  function.  The  automated 
approval  process  will  be  based  on  decision  rules  for  checking  the  content  of  lead 
results  in  investigative  reports.  Ultimately,  this  system  will  require  machine- 
readable  investigative  reports  as  input.  The  purpose  of  the  automated  approval 
component  is  to  identify  adverse  information  of  security  concern  in  order  to 
distinguish  cases  that  may  be  granted  clearance  eligibility  by  the  system  from  those 
that  must  be  reviewed  by  an  adjudicator.  Automated  case  summary  reporting  will 
summarize  relevant  information  about  each  case  and  present  it  to  authorized  users 
with  a  recommendation  for  further  action.  The  objective  of  the  present  study  was  to 
provide  automated  approval  estimates  based  on  a  preliminary  set  of  decision  rules 
for  screening  National  Agency  Check  with  Local  Agency  Checks  and  Credit  Checks 
(NACLC)  investigations. 

APPROACH 

A  previous  study  developed  and  tested  a  large  number  of  conservative  decision 
rules  for  screening  NACLC  investigations,  which  resulted  in  an  unnecessarily 
cautious  approval  rate  of  6%.  In  contrast,  the  present  approach  to  automatic 
approval  employed  fewer,  more  targeted  rules.  The  analysis  began  by  categorizing 
as  ineligible  for  immediate  approval  cases  in  which  the  scope  of  the  investigation 
was  expanded  to  include  additional  leads  and/or  adverse  information  was  identified 
by  the  investigation  provider.  Expanded  investigations  are  likely  to  contain 
information  that  should  be  reviewed  by  an  adjudicator  (e.g.,  potentially 
disqualifying  conditions).  Therefore,  expanded  investigations  generally  should  be 
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ineligible  for  automated  approval.  In  an  operational  ADS  system,  investigations  with 
identified  issues  of  security  concern  and/or  added  coverage  at  this  stage  would  be 
referred  to  an  adjudicator,  while  the  remainder  still  would  be  eligible  for  automatic 
approval. 

A  set  of  additional  “critical  checks”  then  was  applied  to  the  remaining  investigations 
to  ensure  that  only  the  cleanest  cases  would  be  considered  eligible  for  automated 
approval.  As  a  safeguard,  the  decision  rules  for  this  study  screened  certain 
database  fields  that  could  contain  serious  adverse  information  of  security  concern. 
Critical  checks  included  information  from  the  personnel  security  questionnaire, 
national  agency  checks,  local  agency  checks,  and  credit  reports.  Significant  adverse 
information  in  any  of  these  fields  most  likely  would  have  resulted  in  expansion,  but 
the  decision  rules  for  the  ADS  system  should  provide  safeguards  comparable  to 
human  adjudication.  So,  these  additional  checks  were  applied  to  ensure  adequate 
security. 

In  order  to  generate  automated  approval  estimates,  two  random  samples  were 
drawn  from  NACLC  investigations,  for  Secret  and  Confidential  clearance,  closed  by 
the  Defense  Security  Service  (DSS)  in  CY03.  Investigation  data  were  drawn  from  a 
research  copy  of  the  DSS  Case  Control  Management  System  (CCMS)  and  merged 
with  data  from  the  Joint  Personnel  Adjudication  System  (JPAS),  which  contained 
information  about  clearance  eligibility  and  documented  issues.  Cases  with 
incomplete  data  for  the  variables  of  interest  after  merging  CCMS  and  JPAS  tables 
were  omitted  from  the  analysis.  Since  the  results  for  the  two  samples  were  nearly 
identical,  only  results  for  the  second  sample  are  provided  in  this  report. 

RESULTS  AND  DISCUSSION 

The  present  study  evaluated  preliminary  decision  rules  for  automated  screening  of 
NACLC  investigations  and  found  that  approximately  40%  of  applicants  were  eligible 
for  automatic  approval  under  the  reported  rules  because  these  cases  contained 
little  or  no  adverse  information  of  security  concern.  Previous  research  found  that 
only  a  small  percentage  of  cases  were  entirely  devoid  of  adverse  information.  When 
considered  together,  these  studies  provide  a  rationale  and  foundation  for  a  DoD 
ADS  system.  The  DoD  CAFs  process  thousands  of  security  clearance  eligibility 
determinations  every  year.  The  vast  majority  of  applicants  are  granted  access 
eligibility,  and  only  a  small  percentage  of  these  cases  contain  significant  adverse 
information  of  security  concern.  The  use  of  automated  clean-case  screening  could 
provide  significant  cost  savings,  improve  adjudication  timeliness,  and  allow  the 
CAFs  to  focus  human  resources  on  complex  cases  and  those  with  serious  issues. 

DUSD  (ClSsS),  DSS,  PERSEREC  and  other  DoD  components  endorse  a  vision  for  a 
more  integrated,  efficient  and  effective  personnel  security  system  as  part  of  the 
Defense  Information  Systems  for  Security  (DISS)  being  developed  by  DSS.  The 
future  system  will  use  automation  to,  among  other  things,  improve  adverse 
information  detection  and  resolution,  accelerate  investigation  and  adjudication. 
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more  fully  utilize  risk  management  principles,  promote  reciprocity  across  the 
government,  facilitate  accurate  requirements  forecasting,  and  accommodate  surge 
demands  for  clearances.  Notable  changes  will  include  utilizing  the  Automated 
Continuing  Evaluation  System  (ACES)  to  perform  an  approved  variation  of  the 
current  NACLC  investigation,  making  use  of  automation  for  determining  when  to 
request  Special  Interviews  (SPIN)  from  the  Office  of  Personnel  Management  (OPM), 
and  for  evaluating  investigative  results.  ADS  will  be  an  important  component  of  this 
system. 

The  ADS  program  may  be  implemented  more  quickly  by  leveraging  and  repurposing 
ACES  research  completed  at  this  time.  Both  research  programs  seek  to  identify 
cases  of  security  concern,  but  ACES  draws  prompt  attention  to  those  that  pose  the 
greatest  risk,  while  ADS  focuses  on  those  posing  the  least  concern.  Given  the 
similarities  between  the  two  programs,  many  of  the  rules  and  criteria  developed  for 
ACES  should  transfer  in  principle,  if  not  in  the  specifics,  to  ADS. 

Whether  developed  independently  or  in  association  with  ACES,  the  ADS  system  will 
require  machine-readable  data  from  investigation  providers.  Ideally,  all  elements  of 
the  investigative  report  would  be  transmitted  electronically  from  the  provider  in  a 
delimited  format  that  can  be  read  by  a  computer  system.  Also,  summary  codes 
should  detail  the  adjudicative  relevance  of  identified  issues,  and  provide 
information  on  previously  adjudicated  matters  (Kramer,  Crawford,  Ss  Richmond, 
2004;  Richmond  Ss  Timm,  2004;  Leggitt  Ss  Lang,  in  press).  The  minimum 
requirement  for  an  ADS  system  is  that  report  information  be  amenable  to  electronic 
parsing,  search,  and  extraction.  This  requirement  is  likely  to  be  achievable  given 
that  most  standard  NACLC  leads  are  automated  and  personnel  security 
questionnaire  information  is  stored  as  delimited  data  in  e-QlP.  At  some  point,  the 
DISS  may  provide  all  NACLC  data,  except  for  field  lead  information,  via  e-QlP  and 
ACES  record  checks. 

The  methods  and  findings  reported  here  are  intended  to  provide  a  point  of 
departure  for  additional  research  to  validate  and  optimize  decision  rules  and 
approval  estimates  for  different  types  of  investigations.  The  present  research  also 
provides  a  foundation  upon  which  to  build  an  ADS  model  for  SSBls.  The  leads 
conducted  in  NACLC  investigations  also  are  present  in  SSBls,  but  SSBls  contain 
additional  leads  that  must  be  evaluated  by  the  system.  The  foremost  challenge  is  to 
further  develop  methods  for  evaluating  unstructured  text  from  field  leads.  The 
present  study  used  a  keyword  search  to  examine  general  remarks  from  the 
personnel  security  questionnaire.  This  served  the  purpose  of  the  present  study,  but 
SSBls  contain  much  more  unstructured  text  from  subject  and  reference  interviews 
than  do  NACLC  investigations.  Procedures  for  evaluating  this  information  must  be 
further  developed  and  tested. 
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RECOMMENDATIONS 

•  Convene  a  working  group  of  all  stakeholders  (senior  central  adjudication 
facilities,  counterintelligence,  and  security  managers)  to  review  the  preliminary 
decision  rules  and  automated  approval  estimates  from  this  study. 

•  Work  with  investigation  providers  to  obtain  delimited,  machine-readable  input 
for  the  ADS  system. 

•  Extend  and  test  ADS  decision  rules  to  accommodate  Single  Scope  Background 
Investigations  and  periodic  reinvestigations. 

•  Specify  the  data  sources  and  processing  sequence  of  ADS  as  a  component  of  the 
DISS. 
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BACKGROUND 

Management  Initiative  Decision  908  directed  the  Deputy  Under  Secretary  of 
Defense  for  Counterintelligence  and  Security  (DUSD/CISsS)  to  oversee  Joint 
Personnel  Adjudication  System  (JPAS)  improvements  and  e-clearance  integration 
efforts.  Part  of  this  tasking  included  using  automated  procedures  to  accelerate  the 
processing  of  security  clearance  eligibility  determinations.  DUSD  (CISsS)  tasked  the 
Defense  Personnel  Security  Research  Center  (PERSEREC)  to  contribute  to  this 
effort.  One  of  PERSEREC’s  contributions  was  to  initiate  a  research  program  to 
develop  an  Adjudication  Decision  Support  (ADS)  system  that  will  use  automation  to 
improve  adjudication  efficiency  and  effectiveness.  The  present  research  is  the 
second  in  a  series  of  studies  to  develop  preliminary  decision  rules  and  provide 
automated  approval  estimates  for  National  Agency  Check  with  Local  Agency  Checks 
and  Credit  Checks  (NACLC)  investigations. 

Individual  eligibility  for  access  to  classified  information  is  determined  in  accordance 
with  Executive  Order  12968,  Access  to  Classified  Information,  August  4,  1995.  The 
security  clearance  process  consists  of  two  major  functions:  investigation  and 
adjudication.  First,  personnel  security  investigators  collect  information  about  an 
individual’s  personal  history  as  it  relates  to  loyalty,  trustworthiness,  and  reliability. 
This  background  information  serves  as  the  basis  for  determining  clearance 
eligibility.  The  process  of  determining  eligibility  for  access  to  classified  information 
is  referred  to  as  adjudication.  In  the  course  of  adjudication,  information  collected 
during  the  investigation  is  evaluated  against  national  guidelines  (Adjudicative 
Guidelines  for  Determining  Eligibility  for  Access  to  Classified  Information, 

December  29,  2005)  to  determine  if  an  individual  is  an  acceptable  security  risk,  and 
whether  or  not  to  grant  eligibility  for  access. 

The  adjudicative  guidelines  describe  the  adjudication  process  and  indicators  of 
potentially  adverse  information  (i.e.,  disqualifying  conditions).  In  this  context, 
adverse  information  is  information  that  is  contrary  to  granting  eligibility  for  access 
to  classified  information.  Personnel  security  investigations  contain  varying  degrees 
of  adverse  information.  Some  investigations  do  not  contain  adverse  information; 
some  contain  relatively  insignificant  adverse  information  and/or  mitigating 
conditions;  others  include  information  that  may  result  in  denial  or  revocation  of 
access  eligibility.  It  will  be  important  for  an  ADS  system  to  reliably  detect  adverse 
information  of  security  concern  based  on  the  guidelines,  and  to  determine  whether 
cases  should  be  automatically  approved  or  sent  to  an  adjudicator.  The  key  is  to 
determine  appropriate  criteria  for  automating  the  initial  review  process  so  cases 
containing  little  or  no  adverse  information  can  be  automatically  approved  for  access 
without  increasing  risks  to  national  security. 

A  prior  study  by  PERSEREC  (Crawford  da,  Riedel,  1996)  examined  the  costs  and 
benefits  of  clean-case  screening  by  the  Defense  Investigative  Service  (DIS).  The 
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Defense  Security  Service  (DSS;  formerly  DIS)  conducted  personnel  security 
investigations  for  the  Department  of  Defense  (DoD)  prior  to  transferring  this 
function  to  the  Office  of  Personnel  Management  (OPM)  in  February  2005.  The 
authors  compared  DIS  case  analyst  and  adjudicator  ratings  of  the  level  of 
derogatory  information  (clean,  minor  derogatory  information,  major  derogatory 
information)  in  several  different  types  of  investigations  coded  over  a  one-month 
period.  They  also  examined  cost  data  from  the  adjudication  facilities  to  project 
potential  cost  avoidance  with  different  screening  scenarios.  It  was  concluded  that 
DoD  could  save  millions  of  dollars  by  screening  clean  cases  and  cases  with  minor 
derogatory  information  through  a  computer  system,  and  automation  would  pose 
very  little  increase  in  risk.  Furthermore,  automation  should  reduce  clearance 
processing  time,  permit  the  electronic  storage  of  derogatory  information,  and  allow 
collaboration  between  DIS  and  the  adjudication  facilities  to  establish  screening 
criteria. 

The  present  research  builds  on  the  risk  management  perspective  of  Crawford  and 
Riedel  (1996).  Risk  management  is  the  process  of  applying  security 
countermeasures  to  attain  an  acceptable  level  of  risk  at  a  reasonable  cost  (Roper, 
1999).  The  primary  goal  of  the  ADS  program  is  to  develop  a  system  that  will 
automatically  evaluate  completed  personnel  security  investigations  in  accordance 
with  the  Adjudicative  Guidelines  for  Determining  Eligibility  for  Access  to  Classified 
Information  (December  29,  2005),  and  by  using  decision  logic  comparable  to  the 
logic  of  expert  adjudicators  (Youpa,  Marshall-Mies,  Lang  Ss  Carney,  2004). 

DOD  AUTOMATED  PERSONNEL  SECURITY  SYSTEM 

DUSD  (ClSsS),  DSS,  PERSEREC  and  other  DoD  components  endorse  a  vision  for  a 
more  integrated,  efficient  and  effective  personnel  security  system  as  part  of  the 
Defense  Information  Systems  for  Security  (DISS)  being  developed  by  DSS.i  The 
future  system  will  use  automation  to,  among  other  things,  improve  adverse 
information  detection  and  resolution,  accelerate  investigation  and  adjudication, 
more  fully  utilize  risk  management  principles,  promote  reciprocity  across  the 
government,  facilitate  accurate  requirements  forecasting,  and  accommodate  surge 
demands  for  clearances.  Notable  changes  will  include  utilizing  the  Automated 
Continuing  Evaluation  System  (ACES)  to  perform  an  approved  variation  of  the 
current  NACLC  investigation,  making  use  of  automation  for  determining  when  to 
request  Special  Interviews  (SPIN)  from  OPM,  and  for  evaluating  investigative 
results.  ADS  will  be  a  component  of  this  system. 

The  process  envisioned  for  initial  NACLC  investigations  includes  the  following  steps. 
First,  clearance  requests  will  be  submitted  electronically  by  security  managers 
through  an  existing  function  in  the  Joint  Personnel  Adjudication  System  (JPAS). 

1  The  DISS  is  an  enterprise  architecture  system  to  facilitate  a  more  holistic  environment  within 
DSS.  The  system  will  provide  a  secure  software  solution  for  all  aspects  of  the  industrial  security, 
personnel  security,  counterintelligence  and  security  education  training  and  awareness  programs 
by  integrating  several  existing  systems. 
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JPAS  will  electronically  validate  requests  at  the  appropriate  level  and  authorize  the 
OPM  Electronic  Questionnaires  for  Investigations  Processing  (e-QlP)  system  to  allow 
applicant  access.  Applicants  will  complete  the  personnel  security  questionnaire 
using  e-QlP,  and  submit  fingerprints  via  live  scan.  If  applicants  respond 
affirmatively  to  certain  questions  on  the  e-QlP  questionnaire,  the  system  will  ask 
relevant  follow-up  questions.  Digital  signatures  will  be  obtained  and  used, 
eliminating  the  need  for  paper  record  submission  and  retention. 

Next,  ACES  will  acquire  applicant  data  from  e-QlP  and  fingerprint  checks,  and  then 
conduct  additional  record  checks  deemed  to  meet  or  exceed  NACLC  standards. 
Additional  checks  will  include  all  centrally  stored,  electronically  available,  waiver 
and  prescreening  information.  In  addition  to  acquiring  this  information,  ACES  also 
will  perform  preliminary  processing  to  ensure  that  data  are  stored  in  a  format  that 
facilitates  subsequent  analysis.  Data  acquired  by  ACES  and  other  sources  will  be 
evaluated  using  ADS  business  rules  to  determine  which  applicants  qualify  for 
Secret  and  Confidential  level  access  eligibility  (it  is  planned  that  eventually  Single 
Scope  Background  Investigations  for  Top  Secret  access  will  be  screened  by  the  ADS 
system).  The  ADS  component  of  this  system  will  process  electronic  reports  for 
adjudication  (e-RFA)  in  accordance  with  national  guidelines.  The  system  will 
identify  information  that  is  relevant  to  clearance  eligibility  determinations  based  on 
decision  rules  established  by  policy,  senior  adjudicators,  other  personnel  security 
experts,  and  empirical  research  on  past  determinations. 

Cases  that  contain  adverse  information  assessed  to  pose  minimal  risk  will  qualify 
for  automatic  approval.  If  no  potentially  disqualifying  conditions  are  detected  by  the 
system,  the  requested  access  eligibility  will  be  electronically  approved  and  posted  in 
JPAS.  Cases  with  adverse  information  of  security  concern  will  be  evaluated 
electronically  to  assess  whether  an  OPM  SPIN  or  other  review  is  necessary.  In  the 
event  neither  a  SPIN  nor  buffer  cell  review  is  deemed  necessary,  cases  will  be 
adjudicated  by  the  appropriate  adjudication  facility.  This  process  should  greatly 
shorten  case  processing  times  for  the  cleanest  cases,  facilitate  the  initial  review  of 
complex  cases  by  adjudicators,  reduce  the  overall  workload  at  the  CAEs,  increase 
adjudication  consistency,  and  facilitate  case  assignment. 

The  ADS  component  of  the  automated  system  will  include  a  variety  of  controls  to 
safeguard  and  maintain  adjudication  integrity.  Initial  evaluation  of  decision  rules 
for  research  and  development  will  be  conducted  virtually,  in  parallel  to 
determinations  by  adjudicators;  that  is,  all  ADS  system  eligibility  recommendations 
will  be  reviewed  by  adjudicators  to  ensure  that  the  system  is  working  properly.  The 
results  of  virtual  and  manual  adjudication  will  be  compared  and  used  to  modify 
decision  rules,  as  necessary.  Prior  to  final  deployment,  adjudicators  will  continue  to 
periodically  audit  random  samples  of  automatically  approved  cases.  This  procedure 
will  assess  the  ongoing  efficacy  of  ADS  decision  rules  and  improve  security  by 
eliminating  a  priori  certainty  over  which  cases  will  be  reviewed  by  adjudicators. 


3 


INTRODUCTION 


ADS  RESEARCH  PROGRAM 

Current  ADS  research  is  developing  decision  rules,  meant  for  stakeholder  feedback 
and  trial  implementation,  for  automated  approval  of  clearance  eligibility  for 
applicants  with  little  or  no  adverse  information  of  security  concern  in  their 
backgrounds.  The  immediate  objective  is  to  estimate  the  percentage  of  cases  that 
would  be  automatically  approved  for  access  under  different  rule  models.  The 
research  program  is  also  developing  automated  case  summaries,  which  are 
improved  summary  formats  of  the  information  contained  in  electronic  reports  for 
adjudication.  These  summaries  will  document  the  automated  processing  performed, 
and  facilitate  the  manual  adjudication  process  when  automated  approval  is  not 
appropriate.  The  ADS  research  program  is  divided  into  three  stages,  with  each  stage 
corresponding  to  security  clearance  products  across  DoD  military,  civilian,  and 
industrial  applicants: 

1.  National  Agency  Check  with  Local  Agency  Checks  and  Credit  Checks 
(NACLC)  as  well  as  Access  National  Agency  Check  and  Inquiries  (ANACl) 
initial  and  periodic  reinvestigations  for  Confidential  and  Secret  clearances 

2.  Single  Scope  Background  Investigation  Periodic  Reinvestigations  (SSBl-PR) 
for  Top  Secret  (TS)  and  Sensitive  Compartmented  Information  (SCI)  periodic 
reviews 

3.  Single  Scope  Background  Investigations  (SSBl)  for  initial  TS  and  SCI  access 
determinations . 

The  present  study  is  part  of  Stage  1 ,  which  focuses  on  automated  approval 
estimates  for  NACLC  and  ANACl  investigations.  These  types  of  investigations 
contain  relatively  small  amounts  of  unstructured  textual  information.  The 
standards  for  NACLC  and  ANACl  investigations  require  a  completed  personnel 
security  questionnaire,  which  contains  unstructured  text,  as  well  as  national  and 
local  records  checks.  These  investigations  do  not  require  subject  and  reference 
interviews  unless  adverse  information  is  developed  as  part  of  the  standard  inquiry. 
Stage  2  will  extend  the  research  to  SSBl-PRs,  which  include  additional  investigative 
leads.  Stage  3  will  incorporate  the  results  of  earlier  stages  to  test  and  evaluate 
decision  rules  for  initial  SSBls. 

ADS  SYSTEM  COMPONENTS  AND  RESEARCH  OBJECTIVES 

The  ADS  system  is  designed  to  have  two  major  components:  (1)  automated  approval 
processing  logic  and  (2)  an  automated  case  summary  reporting  function.  The 
automated  approval  process  will  consist  of  decision  rules  for  checking  the  presence 
of  potentially  derogatory  content  in  the  lead  results  of  investigative  reports. 
Ultimately,  this  system  will  require  machine-readable  investigative  reports  as  input. 
Automated  Case  Summary  (ACS)  reports  will  be  the  human-readable  output  of  the 
system.  These  reports  will  summarize  relevant  information  about  each  case  and 
present  it  to  authorized  users  with  a  recommendation  for  further  action  made  by 
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the  processing  logic.  The  objective  of  the  present  study  was  to  further  develop  and 
test  the  automated  approval  component  of  the  system. 

The  automated  approval  component  of  the  ADS  system  seeks  to  identify  adverse 
information  of  security  concern  in  investigative  reports  to  distinguish  between  cases 
that  may  be  granted  clearance  eligibility  by  the  system  and  those  that  must  be 
reviewed  by  an  adjudicator.  Youpa,  Marshall-Mies,  Lang,  and  Carney  (2004) 
developed  and  evaluated  an  initial  set  of  decision  rules  for  automated  approval  of 
security  clearances  based  on  NACLC  investigations.  The  initial  set  of  rules  was 
applied  to  a  random  sample  of  FY02  NACLC  investigations  conducted  by  the 
Defense  Security  Service  (DSS)  to  estimate  the  percentage  of  cases  that  would 
qualify  for  automated  approval.  Given  the  intentionally  cautious  rules  that  were 
applied,  the  researchers  found  that  only  6.4%  of  the  cases  in  the  sample  were 
eligible  for  automatic  approval. 

The  present  study  was  an  extension  of  the  Youpa  et  al.  (2004)  project,  and  was 
based  on  risk  management  principles.  The  primary  aim  was  to  estimate  the 
percentage  of  cases  that  would  be  eligible  for  automated  approval  given  a  less 
conservative  yet  reasonable  set  of  decision  rules.  The  research  team  evaluated  a 
select  set  of  rules  for  screening  NACLC  investigations  in  order  to  increase 
automated  approval  estimates,  while  maintaining  an  acceptable  level  of 
effectiveness.  This  study  was,  in  part,  conducted  on  the  premise  that  a  small  and 
focused  set  of  rules  may  be  more  easily  explained,  implemented,  and  optimized 
than  a  large  set  of  complex  rules. 
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APPROACH 

The  present  approach  to  automated  approval  involved  the  following  three  basic 
steps. 

1 .  Identify  investigations  with  added  coverage 

2.  Check  for  adverse  information  in  critically  important  sources 

3 .  Determine  eligibility  for  automated  approval 

Cases  with  added  coverage,  where  the  scope  of  the  investigation  was  expanded  to 
include  additional  leads  and/or  adverse  information  identified  by  the  investigation 
provider  (i.e.,  DSS),  were  considered  ineligible  for  automated  approval.  Although 
most  expanded  investigations  do  not  result  in  an  adverse  action  (denial  or 
revocation) ,  they  are  more  likely  to  contain  derogatory  or  suspicious  information 
that  should  be  reviewed  by  an  adjudicator  (e.g.,  potentially  disqualifying 
conditions).  Based  on  data  from  the  recently  retired  DSS  Case  Control  Management 
System  (CCMS),  it  was  predicted  that  approximately  40  percent  of  the  sampled 
investigations  would  include  additional  coverage,  and  that  the  remainder  would  be 
free  of  significant  adverse  information.  Thus,  although  all  cases  would  be  evaluated 
by  the  ADS  system  and  result  in  an  automated  summary,  only  unexpanded  cases 
would  be  considered  eligible  for  automatic  approval. 

To  safeguard  against  oversights  in  case  expansion,  the  decision  rules  for  this  study 
also  screened  certain  database  fields  that  might  contain  particularly  serious 
adverse  information  of  security  concern.  This  set  of  additional  “critical  checks”  was 
applied  to  the  unexpanded  investigations  to  ensure  that  only  appropriate  cases 
would  remain  eligible  for  automated  approval.  Significant  adverse  information  in 
any  of  these  fields  most  likely  would  have  resulted  in  expansion,  but  an  ADS 
system  should  strive  to  be  accurate  and  comparable  to  human  adjudication. 

The  critical  checks  were  selected  by  the  researchers,  and  included  information  from 
the  personnel  security  questionnaire,  national  agency  checks,  local  agency  checks, 
and  credit  reports.  A  total  of  32  elements  from  the  personnel  security  questionnaire 
were  chosen,  including  a  keyword  search  of  the  general  remarks.  A  CCMS  result 
code  was  used  to  screen  national  agency  and  credit  bureau  checks.  These  codes 
were  based  on  DSS  criteria  for  evaluating  national  agency  and  credit  leads.  While 
the  database  contained  a  similar  result  code  for  field  leads,  including  local  agency 
checks,  an  abundance  of  missing  values  precluded  its  use  in  this  study.  Thus,  a 
computer  program  developed  by  PERSEREC  was  used  to  extract  relevant 
information  from  field  lead  data  in  CCMS.  Among  other  things,  the  program 
identified  the  presence  of  sworn  statements,  attachments,  and/or  additional  leads 
(i.e.,  added  coverage).  For  this  study,  if  any  of  these  items  were  present,  the  case 
was  considered  ineligible  for  automated  approval. 
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PROCEDURE 

In  order  to  assemble  cases  for  this  study,  data  were  extracted  from  research  copies 
of  the  DSS  CCMS  and  the  JPAS.  CCMS  was  used  to  assemble  closed  NACLC 
investigations,  and  JPAS  was  employed  to  obtain  case  dispositions  following 
adjudication.  SPSS  Clementine®  was  the  primary  software  used  to  compile  the 
study  database.  In  addition,  a  software  program  called  ALICE,  developed  by 
PERSEREC,  extracted  field  lead  data.  The  data  extracted  with  ALICE  were  analyzed 
separately  and  merged  with  the  other  data  from  CCMS. 

Two  random  samples  were  drawn  from  NACLC  investigations,  for  Secret  and 
Confidential  clearance,  closed  by  DSS  in  CY03.  Since  the  results  for  the  two 
samples  were  nearly  identical,  only  results  for  the  second  sample  are  provided  in 
this  report.  SPSS  Clementine®  was  used  to  merge  CCMS  tables  containing  leads  for 
the  sampled  cases.  These  data  then  were  merged  with  tables  from  JPAS  that 
contained  information  about  case  dispositions.  This  procedure  provided  the  data 
table  necessary  for  analysis.  The  disposition  data  included  information  about 
clearance  eligibility  and  adjudication  profile  (i.e.,  identified  issues  of  security 
concern,  if  any),  which  were  used  as  outcomes  in  the  analysis.  Cases  with 
incomplete  data  for  the  variables  of  interest  after  merging  CCMS  and  JPAS  tables 
were  omitted  from  the  analysis. 

Much  of  the  data  for  this  study  were  recoded,  aggregated,  and  analyzed  using 
SPSS®  for  Windows  14.0.  For  each  investigative  element  used  in  the  analysis, 
information  regarded  as  acceptable  according  to  the  decision  rules  was  assigned  a 
value  of  0,  and  information  that  could  be  potentially  disqualifying  and  thus  would 
necessitate  review  by  an  adjudicator  was  assigned  a  value  of  1 .  Only  cases  in  which 
all  investigative  elements  were  coded  0,  or  acceptable,  were  considered  eligible  for 
automated  approval  of  clearance  eligibility.  Cases  in  which  any  element  was  coded 
1  were  not  considered  eligible  for  automated  approval  and  would  be  referred  to  an 
adjudicator  for  further  review. 

First  Set  of  Decision  Rules:  Case  Category  Codes 

Two  sets  of  decision  rules  were  used  to  screen  investigations  for  automated 
approval.  The  first  set  applied  existing  CCMS  case  category  codes  to  determine 
whether  or  not  investigations  contained  documented  issues  or  were  expanded.  Case 
category  codes  in  CCMS  indicate,  among  other  things,  the  level  of  clearance 
application,  applicant  community,  the  presence  of  adverse  information,  and 
whether  or  not  there  was  added  coverage  (i.e.,  the  investigation  was  expanded).  The 
codes  in  Table  1  represented  clean  NACLC  investigations  that  did  not  receive 
additional  coverage.  Investigations  with  these  codes  remained  eligible  for  automated 
approval  pending  critical  checks.  The  remaining  cases  were  considered  ineligible  at 
this  stage,  and  would  be  referred  to  an  adjudicator. 
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Table  1 

CCMS  Case  Category  Codes  Used  for  Initial  Screen 

Code^  Description _ 

1 D 1  SECRET  PR  MIL  NON-ISSUE  -  SECURITY  PROGRAM 

lEl  SECRET  PR  CIV  NON-ISSUE  -  SECURITY  PROGRAM 

IFl  SECRET  PR  IND  NON-ISSUE  -  SECURITY  PROGRAM 

2D  1  SECRET  PR  MIL  -  NACLC  NON-ISSUE,  SECURITY  PROGRAM 

2E1  SECRET  PR  CIV  -  NACLC  NON-ISSUE,  SECURITY  PROGRAM 

2F1  SECRET  PR  INC  -  NACLC  NON-ISSUE,  SECURITY  PROGRAM 

2K2  SECRET  NACLC  -  MIL  NON-ISSUE,  SECURITY  PROGRAM 

2L2  SECRET  NACLC  -  CIV  NON-ISSUE,  SECURITY  PROGRAM 

2M2  SECRET  NACLC  -  IND  NON-ISSUE,  SECURITY  PROGRAM 

3D  1  CONFIDENTIAL  PR  MIL  -  NACLC  NON-ISSUE,  SECURITY  PROGRAM 

3E1  CONFIDENTIAL  PR  CIV  -  NACLC  NON-ISSUE,  SECURITY  PROGRAM 

3F1  CONFIDENTIAL  PR  INC  -  NACLC  NON-ISSUE,  SECURITY  PROGRAM 

3K2  CONFIDENTIAL  NACLC  -  MIL  NON-ISSUE,  SECURITY  PROGRAM 

3L2  CONFIDENTIAL  NACLC  -  CIV  NON-ISSUE,  SECURITY  PROGRAM 

3M2  CONFIDENTIAL  NACLC  -  IND  NON-ISSUE,  SECURITY  PROGRAM 


a  Cases  with  these  codes  were  considered  eligible  for  automated  approval  at  this  stage 
of  the  analysis. 

Second  Set  of  Decision  Rules:  Critical  Checks 

The  second  set  of  decision  rules  involved  checking  “critical”  fields.  Table  2  presents 
the  critical  checks  chosen  for  analysis.  Critical  checks  were  employed  as  a 
safeguard  against  approving  investigations  with  potentially  disqualifying  conditions 
that  were  not  identified  by  the  first  set  of  decision  rules  using  case  category  codes. 
These  were  applied  to  Electronic  Personnel  Security  Questionnaire  (EPSQ),  National 
Agency  Check  (NAC),  Field  Lead/ Local  Agency  Check  (LAC),  and  Credit  Bureau 
Check  (CBC)  data. 
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Table  2 

CCMS  Fields  Used  for  Critical  Checks 


CCMS  Field 

Source  Label 

Approval 

Eligibility 

Rule 

EPSQ 

DUAL_CIT_CD 

3.  Dual  Citizenship 

No 

CTZNSHP_TYPE_CD 

3.  Citizenship 

US 

CTZNSHP_CY_CD 

8.  Your  Spouse 

US 

CTZNSHP_CY_CD 

9.  Your  Relatives  and  Associates 

US 

FRGN_BSNS_INTRT_CD 

12.  Your  Foreign  Activities:  Property 

No 

FRGN_EMPL_JST_CD 

13.  Your  Foreign  Activities:  Employment 

No 

FRGN_GVMT_ACT_CD 

14.  Your  Foreign  Activities:  Foreign  Government 

No 

FRGN_PSPRT_DATA_CD 

15.  Your  Foreign  Activities:  Passport 

No 

DISCHRG_DATA_CD 

17.  Your  Military  Record 

No 

MNTL_HLTH_REF 

19.  Your  Medical  Record 

No 

JOB_TRMNTN_IND_CD 

20.  Your  Employment  Record 

No 

FLNY_ARST_OFNS_CD 

21.  Your  Police  Record:  Felony  Offenses 

No 

FRM_EXPL_OFNS_CD 

22.  Your  Police  Record:  Firearms/ Explosives 
Offenses 

No 

PNDG_CHRG_OFNS_CD 

23.  Your  Police  Record:  Pending  Charges 

No 

ALC_DRG_OFNS_CD 

24.  Your  Police  Record:  Alcohol/Drug  Offenses 

No 

MIL_POL_OFNS_CD 

25.  Your  Police  Record:  Military  Court 

No 

MSCLNS_OFNS_CD 

26.  Your  Police  Record:  Other  Offenses 

No 

DRG_ILGL_USE_CD 

27.  Your  Use  of  Illegal  Drugs:  Illegal  Use  of  Drugs 

No 

DRG_SNSTV_PSTN_CD 

28.  Your  Use  of  Illegal  Drugs:  Sensitive  Positions 

No 

DRG_ACTVTY_ILGL_CD 

29.  Your  Use  of  Illegal  Drugs:  Drug  Activity 

No 

ALCH_CNSL_CD 

30.  Your  Use  of  Alcohol 

No 

SEC_DEN_DATA_CD 

32.  Your  Investigation  Record:  Clearance  Actions 

No 

BANKRUPTCY_DATA_CD 

33.  Your  Financial  Record:  Bankruptcy 

No 

WAGE_GAR_DATA_CD 

34.  Your  Financial  Record:  Wage  Garnishments 

No 

REPOS_DATA_CD 

35.  Your  Financial  Record:  Repossessions 

No 

TAX_LI  EN_D  ATA_CD 

36.  Your  Financial  Record:  Tax  Lien 

No 

UNPAID_JUD_DATA_CD 

37.  Your  Financial  Record:  Unpaid  Judgments 

No 

DLDBT180_DATA_CD 

38.  Your  Financial  Delinquencies:  180  Days 

No 

PUB_REC_DATA_CD 

40.  Public  Record  Civil  Court  Actions 

No 

ORG_AFF_M_DATA_CD 

41.  Your  Association  Record:  Membership 

No 

ORG_AFF_A_DATA_CD 

42.  Your  Association  Record:  Activities 

No 

PRMKS 

43.  General  Remarks 

No 

keyword 

hits 

NAC 

NAC_LEAD_RSLT_CD 

NAC  Lead  Result  Code 

Favorable 

LAC  (ALICE  Fields) 

ATTACHMENTS 

Presence  of  Attachment(s) 

Not 

present 

SWORN  STATEMENTS 

Presence  of  Sworn  Statement(s) 

Not 

present 

NON-LAC  LEADS 

Presence  of  Non-LAC  Lead(s) 

Not 

present 

CBC 

NAC_LEAD_RSLT_CD 

NAC  Lead  Result  Code 

Favorable 
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METHODOLOGY 


Investigations  in  the  study  sample  were  based  on  the  DoD  Electronic  Personnel 
Security  Questionnaire  (EPSQ).  The  EPSQ  is  an  electronic  version  of  Standard  Form 
86  Questionnaire  for  National  Security  Positions  (SF-86),  and  consists  of  43  item 
blocks.  Security  clearance  applicants  complete  SF-86  to  initiate  the  investigation 
process.  The  personnel  security  questionnaire  requests  information  such  as 
citizenship,  residence,  education,  employment,  criminal  history,  etc.,  and 
applicants’  responses  help  determine  the  scope  of  the  background  investigation. 
Their  questionnaire  responses  also  are  considered,  along  with  interviews  and 
records  checks,  to  reach  an  appropriate  clearance  eligibility  determination. 

It  should  be  noted  that  with  OPM  now  conducting  investigations,  a  new  electronic 
version  of  the  SF-86  is  being  used  by  DoD  clearance  applicants.  The  OPM  e-QIP 
enables  security  clearance  applicants  to  complete  a  personnel  security 
questionnaire  over  a  secure  Internet  connection.  However,  since  the  EPSQ  was 
used  by  DoD  when  these  data  were  collected,  EPSQ  data  were  analyzed  (see  Table 
2). 

National  Agency  Checks  (NAC)  were  screened  by  using  a  result  code  entered  in 
CCMS.  The  results  of  Credit  Bureau  Checks  also  were  contained  in  the  NAC  lead 
result  field  in  CCMS.  According  to  the  decision  rules  employed  in  this  study,  only 
NACs  and  CBCs  with  result  code  “A”  (Favorable)  were  considered  eligible  for 
automated  approval.  Any  other  result  code  would  necessitate  that  the  case  be 
referred  to  an  adjudicator  for  further  review.  NAC  and  CBC  data  were  considered 
separately  in  this  study. 

Since  result  codes  were  generally  unavailable  for  CY03  field  leads  in  CCMS,  the 
ALICE  program  compiled  a  list  of  cases  with  clean  LACs  and  no  additional  field 
leads  or  attachments.  These  cases  were  considered  eligible  for  automatic  approval. 
The  remaining  cases  were  coded  as  referrals.  In  a  previous  study,  the  researchers 
checked  a  5%  random  sample  of  cases  to  estimate  the  reliability  of  this  procedure 
and  found  no  discrepancies  (Youpa  et  ah,  2004).  Thus,  ALICE  appears  to  provide  a 
reliable  and  accurate  assessment  of  LACs  for  purposes  of  this  evaluation. 

The  final  safeguard  was  a  keyword  search  of  unstructured  text  in  the  EPSQ  General 
Remarks  field.  This  was  done  to  identify  potential  issues  that  were  not  indicated 
elsewhere  on  the  security  questionnaire.  As  detailed  in  Appendix  A,  the 
unstructured  text  was  examined  in  a  series  of  steps.  In  brief,  the  contents  of 
sample  General  Remarks  fields  were  studied  to  generate  a  keyword  list,  which  was 
then  condensed  and  refined  to  increase  its  precision.  When  evaluating  the  final 
sample,  the  presence  of  any  word  on  the  list  resulted  in  exclusion  from  automated 
approval.  While  a  deployed  ADS  system  would  utilize  more  elaborate  text  analytics, 
this  procedure  provided  an  initial  estimate  of  the  impact  of  unstructured  text  on 
overall  automated  approval  rates. 
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RESULTS 

The  investigations  sampled  for  this  study  included  about  5%  (n  =  10,526)  of  the 
roughly  230,000  NACLC  investigations  closed  by  DSS  in  CY03.  With  a  sample  of 
this  size,  there  is  a  99%  confidence  level  that  the  reported  values  are  within  +1.23 
of  the  population  values.  The  sample  consisted  of  8,621  (81.9%)  males  and  1,905 
(18.1%)  females.  Applicants  were  61.8%  (n  =  6,505)  military,  38.0%  (n  =  4,000) 
industrial,  and  0.2%  (n  =  21)  civilian  government  employees. 2  The  sample  contained 
7,054  (67.0%)  NACLC-Secret,  178  (1.7%)  NACLC-Confidential,  3,270  (31.1%) 
Secret-PR,  and  24  (0.2%)  Confidential-PR  investigations. 

CASE  CATEGORY  CODES 

DSS  case  category  codes  were  used  to  initially  eliminate  from  automated  approval 
those  investigations  with  known  issues  of  security  concern  and/or  added  coverage 
(i.e.,  expanded  investigations).  This  sample  contained  2,355  (22.4%)  investigations 
with  case  category  codes  that  represented  identified  issues  and/or  added  coverage. 
A  total  of  8,171  (77.6%)  investigations  were  coded  as  nonissue.  These  nonissue 
investigations  remained  eligible  for  automated  approval,  pending  the  additional 
checks.  Table  3  shows  the  distribution  of  CCMS  case  category  codes  for  these 
investigations. 


2  Investigations  for  civilian  government  employees  were  being  conducted  by  OPM  during  this 
period.  So,  these  types  of  cases  were  underrepresented  in  CCMS,  which  is  a  DoD  database. 
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Table  3 

Distribution  of  CCMS  Case  Category  Codes 


Code 

Label 

Frequency 

Percent 

1D3 

PR  MIL  (NON-ISSUE  -  SECURITY  PROGRAM) 

1 

0.0 

1F3 

PR  IND  (NON-ISSUE  -  SECURITY  PROGRAM) 

1 

0.0 

1S3 

SECRET  PR  MIL  (ISSUE  CASE  -  SUITABILITY) 

2 

0.0 

2D1 

SECRET  PR  MIL  -  NACLC  (NON-ISSUE,  SECURITY  PROGRAM) 

1,584 

15.0 

2DA 

SECRET  PR  MIL  -  NACLC  (NON-ISSUE  -  ADDED  COVERAGE) 

1 

0.0 

2E1 

SECRET  PR  CIV  -  NACLC  (NON-ISSUE,  SECURITY  PROGRAM) 

4 

0.0 

2F1 

SECRET  PR  INC  -  NACLC  (NON-ISSUE,  SECURITY  PROGRAM) 

948 

9.0 

2  FA 

SECRET  PR  IND  -  NACLC  (NON-ISSUE  -  ADDED  COVERAGE) 

17 

0.2 

2K2 

SECRET  NACLC  -  MIL  (NON-ISSUE,  SECURITY  PROGRAM) 

3,625 

34.4 

2  KB 

SECRET  NACLC  -  MIL  (NON-ISSUE,  ADDED  COVERAGE) 

5 

0.0 

2L2 

SECRET  NACLC  -  CIV  (NON-ISSUE,  SECURITY  PROGRAM) 

14 

0.1 

2M2 

SECRET  NACLC  -  IND  (NON-ISSUE,  SECURITY  PROGRAM) 

1,883 

17.9 

2  MB 

SECRET  NACLC  -  IND  (NON-ISSUE,  ADDED  COVERAGE) 

37 

0.4 

2S1 

SECRET  PR  MIL  -  NACLC  (ISSUE  CASE  -  SECURITY) 

67 

0.6 

2S3 

SECRET  PR  MIL  -  NACLC  (ISSUE  CASE  -  SUITABILITY) 

273 

2.6 

2S6 

SECRET  PR  CIV  -  NACLC  (ISSUE  CASE  -  SUITABILITY) 

1 

0.0 

2S7 

SECRET  PR  IND  -  NACLC  (ISSUE  CASE  -  SECURITY) 

71 

0.7 

2S9 

SECRET  PR  INC  -  NACLC  (ISSUE  CASE  -  SUITABILITY) 

208 

2.0 

2SA 

SECRET  PR  MIL  -  NACLC  (ADDED  COVERAGE  FOR  ISSUE  - 
SECURITY) 

1 

0.0 

2SC 

SECRET  PR  MIL  -  NACLC  (ADDED  COVERAGE  FOR  ISSUE  - 
SUITABILITY) 

4 

0.0 

2SG 

SECRET  PR  IND  -  NACLC  (ADDED  COVERAGE  FOR  ISSUE  - 
SECURITY) 

16 

0.2 

2  SI 

SECRET  PR  IND  -  NACLC  (ADDED  COVERAGE  FOR  ISSUE  - 
SUITABILITY) 

7 

0.1 

2V1 

SECRET  NACLC  -  MIL  (ISSUE  CASE  -  SECURITY) 

97 

0.9 

2V3 

SECRET  NACLC  -  MIL  (ISSUE  CASE  -  SUITABILITY) 

774 

7.4 

2V6 

SECRET  NACLC  -  CIV  (ISSUE  CASE  -  SUITABILITY) 

2 

0.0 

2V7 

SECRET  NACLC  -  IND  (ISSUE  CASE  -  SECURITY) 

167 

1.6 

2V9 

SECRET  NACLC  -  INC  (ISSUE  CASE  -  SUITABILITY) 

487 

4.6 

2VA 

SECRET  NACLC  -  MIL  (ADDED  COVERAGE  FOR  ISSUE  - 
SECURITY) 

9 

0.1 

2VC 

SECRET  NACLC  -  MIL  (ADDED  COVERAGE  FOR  ISSUE  - 
SUITABILITY) 

15 

0.1 

2VG 

SECRET  NACLC  -  IND  (ADDED  COVERAGE  FOR  ISSUE  - 
SECURITY) 

18 

0.2 

2VI 

SECRET  NACLC  -  INC  (ADDED  COVERAGE  FOR  ISSUE  - 
SUITABILITY) 

27 

0.3 

3D1 

CONFIDENTIAL  PR  MIL  -  NACLC  (NON-ISSUE,  SECURITY 
PROGRAM) 

1 

0.0 

3F1 

CONFIDENTIAL  PR  INC  -  NACLC  (NON-ISSUE,  SECURITY 
PROGRAM) 

7 

0.1 

3FA 

CONFIDENTIAL  PR  IND  -  NACLC  (NON-ISSUE  -  ADDED 
COVERAGE) 

4 

0.0 

3K2 

CONFIDENTIAL  NACLC  -  MIL  (NON-ISSUE,  SECURITY 
PROGRAM) 

36 

0.3 

3M2 

CONFIDENTIAL  NACLC  -  IND  (NON-ISSUE,  SECURITY 
PROGRAM) 

69 

0.7 

3MB 

CONFIDENTIAL  NACLC  -  IND  (NON-ISSUE,  ADDED 

COVERAGE) 

1 

0.0 

3S1 

CONFIDENTIAL  PR  MIL  -  NACLC  (ISSUE  CASE  -  SECURITY) 

1 

0.0 
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Code 

Label 

Frequency 

Percent 

3S3 

CONFIDENTIAL  PR  MIL  -  NACLC  (ISSUE  CASE  -  SUITABILITY) 

1 

0.0 

3SG 

CONFIDENTIAL  PR  IND  -  NACLC  (ADDED  COVERAGE  FOR 
ISSUE  -  SECURITY) 

1 

0.0 

3S9 

CONFIDENTIAL  PR  INC  -  NACLC  (ISSUE  CASE  -  SUITABILITY) 

2 

0.0 

3SA 

CONFIDENTIAL  PR  MIL  -  NACLC  (ADDED  COVERAGE  FOR 
ISSUE  -  SECURITY) 

1 

0.0 

3SI 

CONFIDENTIAL  PR  IND  -  NACLC  (ADDED  COVERAGE  FOR 
ISSUE  -  SUITABILITY) 

1 

0.0 

3V1 

CONFIDENTIAL  NACLC  -  MIL  (ISSUE  CASE  -  SECURITY) 

1 

0.0 

3V3 

CONFIDENTIAL  NACLC  -  MIL  (ISSUE  CASE  -  SUITABILITY) 

6 

0.0 

3V7 

CONFIDENTIAL  NACLC  -  IND  (ISSUE  CASE  -  SECURITY) 

7 

0.1 

3V9 

CONFIDENTIAL  NACLC  -  INC  (ISSUE  CASE  -  SUITABILITY) 

21 

0.2 

Total 

10,526 

100.0 

CRITICAL  CHECKS 

Critical  checks  were  performed  on  EPSQ,  NAC,  LAC,  and  CBC  fields  in  CCMS.  Table 
4  shows  the  frequency  and  percentage  of  nonissue  investigations  categorized  as 
eligible  and  ineligible  according  to  the  decision  rules  for  ADS  critical  checks.  As 
shown  in  the  table,  50.7%  (n  =  4,140)  of  the  8,171  nonissue  investigations  in  the 
sample  were  eligible  for  automated  approval  after  conducting  critical  checks.  These 
cases  represent  39.3%  of  the  10,526  investigations  in  the  original  sample. 
Considered  separately,  investigations  were  determined  to  be  ineligible  for 
automated  approval  due  to  adverse  information  in  EPSQ  (26.9%),  CBC  (21.5%), 

NAC  (15.3%),  and  LAC  (5.3%)  fields. 
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Table  4 

CCMS  Nonissue  Investigation  Eligibility  for  Automated  Approval  Based  on  Critical 

Checks  (n  =  8,171) 


Field 

Eligible 

Ineligible 

Frequency 

Percent 

Frequency 

Percent 

EPSQ 

5,971 

73.1 

2,200 

26.9 

3.  Dual  Citizenship 

8,158 

99.8 

13 

0.2 

3.  Citizenship 

8,145 

99.7 

26 

0.3 

8.  Spouse  Citizenship 

8,113 

99.3 

58 

0.7 

9.  Relatives  Citizenship 

7,853 

96.1 

318 

3.9 

12.  Foreign  Property 

8,130 

99.5 

41 

0.5 

13.  Foreign  Employment 

8,123 

99.4 

48 

0.6 

14.  Foreign  Contact 

8,086 

99.0 

85 

1.0 

15.  Foreign  Passport 

8,149 

99.7 

22 

0.3 

17.  Military  Discharge 

8,139 

99.6 

32 

0.4 

19.  Medical  Record 

8,154 

99.8 

17 

0.2 

20.  Employment  Record 

7,920 

96.9 

251 

3.1 

21.  Felony  Offenses 

8,107 

99.2 

64 

0.8 

22.  Firearms  Offenses 

8,156 

99.8 

15 

0.2 

23.  Pending  Charges 

8,162 

99.9 

9 

0.1 

24.  Alcohol/ Drug  Offenses 

7,780 

95.2 

391 

4.8 

25.  Military  Court 

8,125 

99.4 

46 

0.6 

26.  Other  Offenses 

7,801 

95.5 

370 

4.5 

27.  Illegal  Use  of  Drugs 

7,963 

97.5 

208 

2.5 

28.  Use  Sensitive  Position 

8,153 

99.8 

18 

0.2 

29.  Drug  Activity 

8,170 

100.0 

1 

0.0 

30.  Use  of  Alcohol 

8,162 

99.9 

9 

0.1 

32 .  Clearance  Actions 

8,149 

99.7 

22 

0.3 

33.  Bankruptcy 

8,107 

99.2 

64 

0.8 

34.  Wage  Garnishments 

8,094 

99.1 

77 

0.9 

35.  Repossessions 

8,155 

99.8 

16 

0.2 

36.  Tax  Lien 

8,151 

99.8 

20 

0.2 

37.  Unpaid  Judgments 

8,163 

99.9 

8 

0.1 

38.  Delinquencies  180  Days 

8,036 

98.3 

135 

1.7 

40.  Public  Records 

8,019 

98.1 

152 

1.9 

4 1 .  Membership 

8,171 

100.0 

0 

0.0 

42.  Activities 

8,171 

100.0 

0 

0.0 

43.  General  Remarks 

7,756 

94.9 

415 

5.1 

NAC  (NAC  Result  Code) 

6,922 

84.7 

1,249 

15.3 

LAC 

7,735 

94.7 

436 

5.3 

CBC  (NAC  Result  Code) 

6,412 

78.5 

1,759 

21.5 

Recommendatiorv^ 

4,140 

50.7 

4,031 

49.3 

“  The  percentage  of  cases  eligible  for  automated  approval  from  the  original  sample  of 
10,526  investigations  is  39.3%. 


JPAS  DISPOSITIONS  OF  CCMS  NONISSUE  INVESTIGATIONS 

JPAS  dispositions  of  CCMS  nonissue  investigations  were  checked  to  assess  the 
validity  of  the  decision  rules  used  in  this  study.  The  analysis  included  examination 
of  JPAS  issue  distribution  and  documented  clearance  eligibility.  The  distribution  of 
issues  recorded  in  JPAS  for  CCMS  nonissue  investigations  is  presented  in  Table  5. 
As  can  be  seen  in  the  table,  79.6%  (n  =  6,501)  of  CCMS  nonissue  investigations  in 
this  sample  had  no  issues  documented  in  JPAS  at  the  time  of  the  study.  The 
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remaining  20.4%  (n  =  1,670)  of  the  cases  had  from  one  to  five  issue  categories 
documented  by  adjudicators.  A  total  of  891  (10.9%)  of  the  investigations  with 
documented  issues  had  only  one  identified  security  concern  (i.e.,  adjudicative 
guideline). 


Table  5 

JPAS  Issue  Distribution  for  CCMS  Nonissue  Investigations 


JPAS  Issue  Categories 

Frequency 

Percent 

Cumulative  Percent 

0 

6,501 

79.6 

79.6 

1 

891 

10.9 

90.5 

2 

461 

5.6 

96.1 

3 

220 

2.7 

98.8 

4 

81 

1.0 

99.8 

5 

17 

0.2 

100.0 

Total 

8,171 

100.0 

Clearance  eligibility  for  CCMS  nonissue  cases  was  retrieved  from  a  research  copy  of 
JPAS.  The  overwhelming  majority  of  cases  in  the  study  sample  contained  positive 
adjudication  actions.  A  very  small  percentage  had  what  would  be  considered 
unfavorable  determinations.  Table  6  shows  the  distribution  of  outcomes  for  CCMS 
nonissue  investigations.  As  can  be  seen  in  the  table,  97.5%  (n  =  7,967)  of  the  cases 
were  eligible  for  Secret  clearance  after  adjudication.  A  total  of  99.2%  (n  =  8,103)  of 
the  sampled  cases  received  favorable  eligibility  determinations. 


Table  6 

Clearance  Eligibility  for  CCMS  Nonissue  Investigations 


Code 

Eligibility 

Frequency 

Percent 

3 

Pending  Reply  to  Statement  of  Reasons 

5 

0.1 

C 

Confidential 

58 

0.7 

D 

Denied 

4 

0.0 

J 

No  Determination  Made 

11 

0.1 

P 

Interim  Top  Secret 

49 

0.6 

Q 

Favorable 

5 

0.1 

R 

Revoked 

3 

0.0 

S 

Secret 

7,967 

97.5 

u 

Interim  SCI 

23 

0.3 

V 

SCI  -  DCID  6/4 

1 

0.0 

z 

Loss  of  Jurisdiction 

45 

0.6 

Total 

8,171 

100.0 

Table  7  provides  a  cross-tabulation  between  ADS  eligibility  for  automated  approval 
and  JPAS  issues  for  CCMS  nonissue  investigations.  A  total  of  3,780  (91.3%) 
investigations  considered  eligible  for  automated  approval  based  on  preliminary  ADS 
decision  rules  had  no  documented  issues  in  JPAS.  In  contrast,  360  (8.7%) 
investigations  that  were  deemed  eligible  for  automated  approval  had  documented 
issues,  and  were  considered  possible  misses  in  the  sense  that  some  relevant 
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adverse  information  may  not  have  been  detected,  given  the  decision  rules  used  in 
this  study.  Of  investigations  considered  ineligible  for  automated  approval,  2,669 
(66.2%)  were  clean,  and  1,362  (33.8%)  had  documented  issues.  Cases  deemed 
ineligible  for  automated  approval  by  ADS  decision  rules,  but  that  had  no 
documented  issues  in  JPAS,  were  considered  possible  false  alarms.  In  an 
operational  system,  misses  are  more  problematic  than  false  alarms  because  misses 
represent  cases  with  potential  security  concerns  that  could  be  approved 
automatically.  Since  the  primary  purpose  of  this  study  was  to  obtain  automated 
approval  rate  estimates,  the  presence  of  possible  misses  is  inconsequential.  As  part 
of  future  research  and  development,  decision  rules  can  be  adjusted  to  reduce  the 
miss  rate  (i.e.,  detect  additional  adverse  information),  if  previously  undetected 
adverse  content  is  deemed  to  require  attention.  Examples  of  possible  undetected 
adverse  information  from  this  study  are  presented  in  Appendix  B. 


Table  7 

Eligibility  for  Automated  Approval  by  JPAS  Issues  Cross  Tabulation 


ADS  Eligible 

ADS  Ineligible 

Row  Total 

Frequency 

Percent 

Frequency 

Percent 

Frequency 

Percent 

JPAS  Clean 

3,780 

91.3 

2,669 

66.2 

6,449 

78.9 

JPAS  Issues 

360 

8.7 

1,362 

33.8 

1,722 

21.1 

Column  Total 

4,140 

100.0 

4,031 

100.0 

8,171 

100.0 

JPAS  DISPOSITIONS  OF  ADS  ELIGIBLE  WITH  ISSUES  CASES 

The  cross-tabulation  between  ADS  eligibility  for  automated  approval  and  the 
presence  of  documented  issues  in  JPAS  (see  Table  7)  showed  that  360  cases  were 
ADS-eligible,  but  had  issues  recorded  by  adjudicators.  Table  8  shows  the 
distribution  of  issue  categories  for  these  cases.  As  can  be  seen  in  the  table,  the 
majority  (72.8%)  had  one  documented  issue  category,  while  the  remaining  cases 
(27.2%)  contained  multiple  issues. 


Table  8 

Issue  Distribution  for  ADS  Eligible  with  JPAS  Issues  Cases 


JPAS  Issue  Categories 

Frequency 

Percent 

Cumulative  Percent 

1 

262 

72.8 

72.8 

2 

72 

20.0 

92.8 

3 

18 

5.0 

97.8 

4 

7 

1.9 

99.7 

5 

1 

0.3 

100.0 

Total 

360 

100.0 

The  360  potentially  “missed”  cases  were  explored  further  by  examining  the  content 
of  each  investigation  in  CCMS.  Most  of  these  cases  contained  only  minor  adverse 
information,  such  as  isolated,  small  dollar  collections,  single  30  to  90  days  past  due 
accounts,  and  petty  traffic  violations.  A  minority  of  cases  contained  more  serious 
information  about  misdemeanor  arrests,  extensive  foreign  travel,  and  bad  debt  in 
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excess  of  $2,000.  Two  personnel  security  experts^  independently  examined  the  data 
and  agreed  that  approximately  20%  of  the  potentially  missed  cases  contained 
adverse  information  of  security  concern.  The  experts  disagreed  with  each  other 
about  the  status  of  18%  of  the  cases.  The  remaining  cases  (62%)  were  determined 
to  not  contain  significant  adverse  information. 

Table  9  shows  the  clearance  eligibility  documented  in  JPAS  for  cases  that  were 
ADS-eligible  with  issues.  None  of  these  cases  received  unfavorable  eligibility 
determinations  (e.g.,  denial  or  revocation).  Of  the  360  cases  with  issues  noted  in 
JPAS,  99.4%  (n  =  358)  received  favorable  eligibility  determinations.  The  remaining 
0.6%  (n  =  2)  were  “Loss  of  Jurisdiction”  for  which  the  applicants  no  longer  required 
access  eligibility  or  had  transferred  from  the  component  sponsoring  the  clearance 
request. 

Table  9 

Clearance  Eligibility  for  ADS  Eligible  with  JPAS  Issues  Cases 


Code 

Eligibility 

Frequency 

Percent 

s 

Secret 

358 

99.4 

z 

Loss  of  Jurisdiction 

2 

0.6 

Total 

360 

100.0 

In  addition,  a  subset  of  ADS  eligible,  “clean”  cases  (n  =  114)  were  examined  to 
compare  the  level  of  adverse  information  in  the  two  groups.  Despite  the  presence  of 
many  clean  investigations,  there  were  cases  with  comparable  levels  of  unfavorable 
information  in  the  clean  subset.  The  same  two  personnel  security  experts 
independently  coded  17%  of  the  “clean”  subset  as  containing  adverse  information. 
They  disagreed  amongst  themselves  about  9%  of  the  cases,  and  the  remainder 
(74%)  was  considered  clean. 

These  results  demonstrate  inconsistency  in  the  application  of  JPAS  adjudication 
profile  codes,  and  cast  doubt  on  the  utility  of  these  codes  for  research.  Prior 
personnel  security  research  (Crawford  Ss  Trent,  1987)  noted  problems  with  the 
identification  of  issue  cases.  Some  coding  ambiguities  may  result  from  the  ability  to 
quickly  resolve  ambiguous  content,  adverse  information  present  in  the  initial 
investigation  that  is  not  considered  to  require  further  attention,  and  possible 
resistance  to  the  use  of  the  issue  case  label  (for  fear  of  exaggerating  the  severity  of 
minor  derogatory  content) . 

The  profile  codes  indicate  the  adequacy  of  the  present  decision  rules  for  detecting 
adverse  information.  A  small  percentage  (38%  of  360  cases,  or  1.59%  of  8,171 
nonissue  cases)  of  the  evaluated  sample  would  have  received  favorable 
determinations  with  a  mild  to  moderate  level  of  derogatory  content,  but  none  of 
these  was  actually  denied  eligibility  by  adjudicators.  (See  examples  in  Appendix  B.) 


3  Both  expert  consultants  for  this  study  have  worked  for  many  years  in  counterintelligence  and 
security  for  DoD  and  the  intelligence  community.  They  currently  conduct  personnel  security 
investigations. 
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Personnel  security  stakeholders  must  provide  feedback  about  the  appropriateness 
of  these  standards  and  criteria,  on  whether  the  rules  and  permitted  derogatory 
content  entail  acceptable  risks.  Furthermore,  a  deployed  ADS  system  will  require 
periodic  assessments  and  optimization  of  decision  rules  to  adequately  detect  all 
necessary  adverse  information  of  security  concern. 
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CONCLUSION 

The  present  study  evaluated  preliminary  decision  rules  for  automated  screening  of 
NACLC  investigations  and  found  that  approximately  40%  of  applicants  were  eligible 
for  automatic  approval  under  the  reported  rules  because  these  cases  contained 
little  or  no  adverse  information  of  security  concern.  Previous  research  found  that 
only  a  small  percentage  of  cases  were  entirely  devoid  of  adverse  information  (Youpa, 
Marshall-Mies,  Lang,  Ss  Carney,  2004).  Together,  these  studies  provide  a  rationale 
and  foundation  for  a  DoD  ADS  system.  The  DoD  CAFs  process  thousands  of 
security  clearance  eligibility  determinations  every  year.  The  vast  majority  of 
applicants  are  granted  access  eligibility,  and  only  a  small  percentage  of  these  cases 
contain  significant  adverse  information  of  security  concern.  The  use  of  automated 
clean  case  screening  could  provide  significant  cost  savings,  improve  adjudication 
timeliness,  and  allow  the  CAFs  to  focus  human  resources  on  complex  cases  and 
those  with  serious  issues  (Crawford  85  Riedel,  1996). 

The  ADS  system  may  be  implemented  faster  by  leveraging  and  repurposing  ACES 
research  completed  at  this  time.  Both  research  programs  seek  to  identify  cases  of 
security  concern,  but  ACES  draws  prompt  attention  to  those  that  pose  the  greatest 
risk  (Chandler  85  Timm,  2002),  while  ADS  focuses  on  those  posing  the  least 
concern.  Given  the  similarities  between  the  two  programs,  many  of  the  rules  and 
criteria  developed  for  ACES  (e.g..  Chandler  85  Rome,  2005)  should  transfer  in 
principle,  if  not  in  the  specifics,  to  ADS.  The  most  important  difference  is  that  ACES 
looks  at  recent  and  more  troubling  security-related  developments  (Chandler  85 
Timm,  2002),  while  ADS  must  consider  whether  single  modest  concerns  should  be 
reviewed,  whether  patterns  of  minor  concerns  should  be  reviewed,  and  the 
enduring  relevance  of  previously  adjudicated  issues.  Furthermore,  ACES  does  not 
perform  as  extensive  an  examination  of  unstructured  text  fields  for  potentially 
derogatory  content  as  would  be  necessary  for  ADS.  In  sum,  ADS  will  require  more 
restrictive  approval  criteria  than  ACES. 

In  addition  to  being  more  conservative  than  ACES,  the  ADS  program  has  a  different 
measure  of  success.  The  ACES  program  assesses  its  success  in  terms  of  rules  that 
provide  information  of  immediate  use  to  case  handlers  or  adjudicators  (Chandler  85 
Timm,  2002).  ACES  must  provide  actionable  information  for  users  to  find  value  in 
reviewing  its  output.  In  contrast,  the  cases  automatically  approved  by  ADS  would 
only  be  reviewed  for  auditing  purposes  or  when  ADS  findings  are  pertinent  to  a 
subsequent  personnel  security  assessment.  ADS  is  primarily  concerned  with 
reducing  the  adjudication  workload,  without  substantially  increasing  security  risk. 
The  decision  rules  for  automated  clean  case  screening  are  different  than  those 
appropriate  for  continuing  evaluation.  Moreover  ADS  also  must  record  even  minor 
adverse  information  for  auditing  purposes.  Therefore,  ADS  rules  will  require 
extensive  stakeholder  feedback  parallel  to  that  provided  for  ACES  decision  rules 
(e.g..  Chandler  85  Rome,  2005). 
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One  limitation  of  the  present  study  is  that  it  made  use  of  data  structures  from  the 
discontinued  DoD  EPSQ  and  CCMS  applications,  while  current  investigations  are 
conducted  by  OPM  with  e-QlP  and  the  Personnel  Investigations  Processing  System 
(PIPS).  Leggitt  and  Lang  (in  press)  examined  the  structure  of  OPM  electronic  case 
records  and  found  they  are  not  well  suited  for  automation,  but  when  used  with 
other  DoD  data  sources  they  may  contribute  to  an  effective  ADS  system.  Any  future 
ADS  system  will  utilize  information  from  the  most  current  personnel  databases.  It 
is  expected  that  input  for  an  ADS  system  may  initially  come  from  OPM,  then 
eventually  from  the  DISS.  The  impact  of  other  data  sources  on  automated  approval 
estimates  is  unknown. 

A  problem  of  both  CCMS  and  OPM  data  is  that  issue  coding  is  too  imprecise  and 
inconsistent  for  use  in  an  ADS  system.  The  present  study  used  CCMS  case  category 
codes  as  an  initial  screen  of  adverse  content,  and  documented  issue  categories  in 
JPAS  were  utilized  as  an  outcome  measure.  Inconsistencies  in  these  data  may  have 
contributed  to  possible  system  misses  and  false  alarms  in  the  evaluation  of  the 
decision  rules.  In  concurrence  with  Crawford  and  Trent  (1987),  the  present  study 
found  that  CCMS  investigations  flagged  as  containing  issues  did  not  necessarily 
include  significant  adverse  information  of  security  concern.  PERSEREC  has  also 
examined  the  case  summary  codes  in  use  by  OPM  (Kramer,  Crawford,  Ss  Richmond, 
2004;  Richmond  85  Timm,  2004;  Leggitt  85  Lang,  in  press),  and  they  present  even 
greater  ambiguity.  Flawed  codes  would  not  block  the  creation  of  an  effective 
automated  system,  but  the  system  could  achieve  the  greatest  security  and 
efficiency  through  reliable  codes.  If  reliable  summary  codes  are  unavailable,  an  ADS 
system  would  have  to  focus  on  raw  data. 

The  ADS  system  will  require  machine-readable  data  from  the  investigation  provider. 
Ideally,  all  elements  of  the  investigative  report  would  be  transmitted  electronically 
from  the  provider  in  a  delimited  format  that  can  be  read  by  a  computer  system. 

Also,  summary  codes  should  detail  the  adjudicative  relevance  of  identified  issues, 
and  provide  information  on  previously  adjudicated  matters  (Kramer,  Crawford,  85 
Richmond,  2004;  Richmond  85  Timm,  2004;  Leggitt  85  Lang,  in  press).  The 
minimum  requirement  for  an  ADS  system  is  that  report  information  be  amenable  to 
electronic  parsing,  search,  and  extraction.  This  requirement  is  likely  to  be 
achievable  given  that  most  standard  NACLC  leads  are  automated  and  personnel 
security  questionnaire  information  is  stored  as  delimited  data  in  e-QlP.  At  some 
point,  the  DISS  may  provide  all  NACLC  data,  except  for  field  lead  information,  via  e- 
QIP  and  ACES  record  checks. 

The  methods  and  findings  reported  here  are  intended  to  provide  a  point  of 
departure  for  additional  research  to  validate  and  optimize  decision  rules  and 
approval  estimates  for  different  types  of  investigations.  Another  step  in  ADS 
research  will  be  to  evaluate  decision  rules  for  SSBls.  The  present  research  provides 
a  foundation  upon  which  to  build  an  ADS  model  for  SSBls.  The  leads  conducted  in 
NACLC  investigations  also  are  present  in  SSBls,  but  SSBls  contain  additional  leads 
that  must  be  evaluated  by  the  system.  The  foremost  challenge  is  to  further  develop 
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methods  for  evaluating  unstructured  text  from  field  leads.  The  present  study  used  a 
keyword  search  to  examine  general  remarks  from  the  personnel  security 
questionnaire.  This  served  the  purpose  of  the  present  study,  but  SSBls  contain 
much  more  unstructured  text  from  subject  and  reference  interviews  than  do 
NACLC  investigations.  Procedures  for  evaluating  this  information  must  be  further 
developed  and  tested.  In  addition,  adjudication  experts  and  other  personnel 
security  stakeholders  should  provide  feedback  on  ADS  screening  criteria  and 
procedures. 

RECOMMENDATIONS 

•  Convene  a  working  group  of  all  stakeholders  (senior  central  adjudication 
facilities,  counterintelligence,  and  security  managers)  to  review  the  preliminary 
decision  rules  and  automated  approval  estimates  from  this  study. 

•  Work  with  investigation  providers  to  obtain  delimited,  machine-readable  input 
for  the  ADS  system. 

•  Extend  and  test  ADS  decision  rules  to  accommodate  Single  Scope  Background 
Investigations  and  periodic  reinvestigations. 

•  Specify  the  data  sources  and  processing  sequence  of  ADS  as  a  component  of  the 
DISS. 
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PRELIMINARY  ANALYSIS  OF  UNSTRUCTURED  TEXT  IN  EPSQ 

GENERAL  REMARKS 

The  purpose  of  the  Adjudication  Decision  Support  (ADS)  system  is  to  detect  adverse 
information  of  security  concern  in  completed  personnel  security  investigations,  and 
to  determine  which  cases  are  eligible  for  automated  approval  of  clearance  eligibility. 
The  ADS  system  will  be  designed  to  exclude  from  automatic  approval  cases  that 
contain  significant  adverse  information,  and  refer  them  for  adjudication.  While 
NACLC  investigations,  which  were  the  focus  of  this  study,  consist  primarily  of 
tightly  organized  data,  they  also  contain  unstructured  text  from  remarks  entered  on 
the  Electronic  Personnel  Security  Questionnaire  (EPSQ)."^  Moreover,  adverse 
information  sometimes  appears  only  in  EPSQ  remarks.  So,  the  ADS  system  must  be 
able  to  reliably  detect  such  content. 

The  EPSQ  includes  at  least  29  remarks  fields  for  applicant  explanations.  The 
majority  of  EPSQ  remarks  provide  an  opportunity  to  explain  affirmative  responses 
to  issue-relevant,  filter  items.  In  keeping  with  the  risk-management  approach  of 
this  study,  most  of  the  EPSQ  remarks  fields  were  determined  to  be  relatively 
inconsequential  because  they  were  directly  related  to  filter  questions.  The  General 
Remarks  field  is  an  exception  because  it  is  not  linked  directly  to  any  other  item  and 
may  include  a  diverse  range  of  content.  As  described  in  the  body  of  this  report, 
result  codes  for  case  expansion  plus  additional  critical  checks  should  have  detected 
most  relevant  derogatory  content.  The  EPSQ  General  Remarks  field  was  included  as 
a  critical  check  in  this  research  because  it  could  contain  unique  information. 

KEYWORD  MATCHING 

Unstructured  text  poses  a  special  challenge  to  designing  an  ADS  system  because 
natural  languages  use  complex  rules  that  are  difficult  to  address  with  computers. 

As  an  initial  step,  keyword  matching  was  explored  as  a  way  to  detect  adverse 
content  in  unstructured  text  from  EPSQ  General  Remarks.  A  keyword  list  was 
developed  to  include  those  words  that  primarily  indicate  the  presence  of  adverse 
information,  and  to  minimize  the  selection  of  innocuous  material. 

Keyword  matching  is  one  of  the  simplest  methods  for  analyzing  unstructured  text. 
Essentially,  a  list  of  target  words  is  generated,  and  then  a  computer  program  uses 
the  list  to  find  matching  character  strings  in  a  body  of  text.  The  list  of  words  can  be 
optimized  based  on  observed  outcomes  with  test  documents.  Furthermore,  the 
method  is  relatively  transparent,  so  interested  parties  can  easily  see  what  was 
done.  For  these  reasons,  keyword  matching  was  chosen  as  the  initial  approach  to 
evaluating  unstructured  text  in  the  ADS  research  program. 


4  The  EPSQ  is  an  electronic  version  of  Standard  Form  86.  The  EPSQ  was  being  used  by  DoD 
security  clearance  applicants  during  the  period  these  data  were  collected.  DoD  applicants 
currently  are  using  e-QlP. 
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METHOD 

PERSEREC  developed  a  computer  program,  Keyword  Search  (KWS),  to  facilitate 
assessment  of  potential  keywords  for  this  study.  KWS  was  used  to  search  a 
database  for  specified  character  strings.  The  program  then  presented  target 
detection  rates  per  investigation  and  as  independent  character  string  elements, 
initially,  KWS  was  used  to  evaluate  potential  keywords  against  a  development 
sample  of  CY03  NACLC  investigations  from  the  Defense  Security  Service  Case 
Control  Management  System.  A  sample  of  4,457  cases  with  a  total  of  13,105 
remarks  was  examined.  This  sample  included  all  EPSQ  remarks  fields,  such  as 
those  pertaining  to  employment,  education,  and  military  service. 

Potential  keywords  were  found  by  inspecting  remarks  that  previously  were 
determined  by  one  of  the  authors  to  contain  issue-relevant  content.  Relevant  terms 
were  extracted  from  these  fields.  The  criteria  for  establishing  issue-relevance  were 
based  on  the  Adjudicative  Guidelines  for  Determining  Eligibility  for  Access  to 
Classified  Information,  training  in  adjudication  from  the  Defense  Security  Service 
Academy  (DSSA),  and  prior  PERSEREC  research  (Kramer,  Crawford,  da,  Richmond, 
2005;  Kramer,  Crawford,  Ss  Bosshardt,  2005,  Kramer;  Crawford,  Heuer,  Ss  Hagen, 
2001).  The  keyword  list  used  in  this  study  also  incorporated  word  lists  previously 
generated  by  PERSEREC  subject-matter  experts  for  other  related  projects. 

A  final  list  of  keywords  resulted  from  several  iterations  of  analysis  using  KWS. 
Keywords  were  tested  to  estimate  their  utility  in  detecting  adverse  information  of 
security  concern.  All  remarks  in  the  development  sample  were  read  and  categorized 
for  issue-relevance  according  to  previously  discussed  criteria.  A  preliminary  list  of 
keywords  was  reduced  through  an  iterative  process  of  scrutinizing  the  list  for  false 
alarms,  including  and  excluding  individual  keywords,  and  running  the  analysis 
again.  A  false  alarm  was  defined  as  a  keyword  found  in  a  remark  without  issue 
relevance. 

Two  test  samples  of  NACLC  EPSQ  General  Remarks  (hereafter  referred  to  as 
Samples  1  and  2)  were  then  used  to  evaluate  the  final  keyword  list.  Only  EPSQ 
General  Remarks  fields  were  used  for  this  part  of  the  keyword  analysis.  Sample  1 
consisted  of  7,954  cases  with  995  remarks,  and  Sample  2  consisted  of  8,171  cases 
with  1,003  remarks.  The  body  of  this  report  contains  additional  information  about 
the  investigations  in  Sample  2.  KWS  was  utilized  to  perform  this  evaluation.  The 
program  provided  frequencies  of  hits,  misses,  false  alarms,  and  correct  rejections  to 
assess  the  utility  of  keywords  for  identifying  issue-relevant  remarks. 
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RESULTS 

Analysis  of  keywords  using  the  development  sample  resulted  in  a  final  list  of  442 
words  (see  Appendix  C) .  This  list  was  further  evaluated  by  searching  General 
Remarks  from  Samples  1  and  2.  Based  on  analysis  by  KWS,  the  issue-relevant  hit 
rate  was  35%  in  both  samples,  and  the  correct  rejection  rate  was  56%  and  57%  for 
Samples  1  and  2  respectively,  resulting  in  a  correct  identification  rate  of  91%  for 
Sample  1,  and  92%  for  Sample  2.  False  alarms  ranged  from  5%  to  7%,  and  2%  to 
3%  of  issue-relevant  remarks  in  the  two  samples  were  missed.  The  percentages  of 
incorrectly  identified  remarks  were  9%  and  8%  in  Samples  1  and  2  respectively  (see 
Table  A-1). 


Table  A-1 

Results  of  General  Remarks  Ke3nvord  Analysis 


Sample  1  (n=995) 

Sample  2  (n 

=1003] 

KWS  Analysis 

Frequency 

Percent 

KWS  Analysis 

Frequency 

Percent 

Hits 

354 

35 

Hits 

354 

35 

Correct  Rejections 

553 

56 

Correct  Rejections 

564 

57 

Total  Correctly 
Identified 

907 

91 

Total  Correctly 
Identified 

918 

92 

False  Alarms 

73 

07 

False  Alarms 

49 

05 

Misses 

15 

02 

Misses 

36 

03 

Total  Incorrectly 
Identified 

88 

09 

Total  Incorrectly 
Identified 

85 

08 

Issue-relevant  remarks  that  were  not  detected  by  the  keyword  search  (i.e.,  misses) 
were  read  and  categorized  by  adjudicative  guideline.  As  can  be  seen  in  Table  A-1, 
there  were  15  (2%)  misses  in  Sample  1  and  36  (3%)  misses  in  Sample  2.  Most  of  the 
missed  content  fell  within  the  Foreign  Influence  guideline.  Adverse  information 
pertaining  to  foreign  influence  proved  difficult  to  detect  in  these  cases  because 
words  such  as  “family,”  “spouse,”  and  “mother,”  which  may  identify  relatives  living 
outside  the  United  States  or  registered  aliens  within  the  United  States  also  tend  to 
inflate  the  false  alarm  rate.  Nevertheless,  cases  with  missed  remarks  could  have 
been  detected  as  a  result  of  one  or  more  other  critical  checks.  In  Sample  1,  8  (53%) 
of  the  cases  with  missed  issue-relevant  General  Remarks  were  flagged  by  other 
critical  checks.  Likewise,  in  Sample  2,  27  (75%)  of  the  keyword  misses  were  caught 
by  other  checks. 

Keywords  with  at  least  5%  hits  in  each  sample  are  displayed  in  Table  A-2.  As  can 
be  seen  in  the  table,  these  words  had  very  similar  hit  rates  in  both  samples.  For 
example,  the  word  “paid”  had  a  hit  rate  of  38.69%  and  37.04%  in  Samples  1  and  2, 
respectively.  About  half  of  the  keywords  in  Table  A-2  relate  to  financial  issues  such 
as  bankruptcy,  credit  problems,  and  delinquent  accounts.  The  other  half  relate  to 
misdemeanor  criminal  issues  such  as  speeding  tickets,  payment  of  fines,  and  other 
offenses.  Marijuana  use,  in  most  cases,  was  self-reported  by  subjects  as 
experimental,  and  did  not  involve  criminal  charges  or  proceedings. 
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Table  A-2 

Comparison  of  Ke3nvord  Hits  between  Test  Samples 


Sample  1 

Sample  2 

Keyword 

Issue- 

Relevant 

Hits 

Issue- 
Relevant 
Hit  Rate  (%) 

Keyword 

Issue- 

Relevant 

Hits 

Issue- 

Relevant 

Hit  Rate  (%) 

paid 

146 

39.57 

paid 

149 

38.21 

$ 

136 

36.86 

$ 

144 

36.92 

fine 

126 

34.15 

fine 

129 

33.08 

speed 

108 

29.27 

speed 

107 

27.44 

ticket 

67 

18.16 

ticket 

78 

20.00 

police 

60 

16.26 

court 

67 

17.18 

court 

49 

13.28 

police 

57 

14.62 

pay 

40 

10.84 

pay 

44 

11.28 

credit 

40 

10.84 

speeding  ticket 

38 

9.74 

speeding  ticket 

31 

8.40 

credit 

32 

8.21 

fail 

31 

8.40 

fail 

32 

8.21 

marijuana 

29 

7.86 

debt 

27 

6.92 

debt 

27 

7.32 

marijuana 

22 

5.64 

offense 

22 

5.96 

dismiss 

22 

5.64 
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CONCLUSION 

Unstructured  text  in  EPSQ  general  remarks  was  examined  as  a  safeguard  in  the 
overall  risk-management  model  of  this  study.  Specifically,  keyword  matching  was 
used  as  a  critical  check  and  to  begin  to  estimate  the  unique  impact  of  unstructured 
text  on  automated  approval  rates  in  an  ADS  system.  More  sophisticated  text 
analysis  would  be  used  to  reduce  errors  in  a  deployed  system,  but  overall 
automated  approval  rates  are  unlikely  to  be  greatly  affected. 

The  present  keyword  list  was  primarily  generated  through  the  analysis  of  actual 
text  in  NACLC  investigations.  It  is,  therefore,  tuned  to  detect  the  most  frequently 
occurring  adverse  information  rather  than  all  possible  derogatory  content  in  EPSQ 
remarks.  Infrequent  topics,  by  their  nature,  may  have  only  a  negligible  impact  on 
error  and  approval  rates,  but  may  be  important  to  include  in  ADS  keyword  lists. 

The  criteria  for  keyword  inclusion  must  be  discussed  with  personnel  security 
stakeholders. 

Future  ADS  research  will  address  Single  Scope  Background  Investigations  (SSBl). 
SSBls  include  far  more  unstructured  text  than  NACLC  investigations  and  call  for  a 
more  comprehensive  approach  to  text  analysis.  The  ADS  program  will  use  the 
present  findings  as  a  foundation  for  future  research.  This  effort  may  include 
established  keyword  lists  and  appropriate  commercial  text  analytic  software. 
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EXAMPLES  OF  ADVERSE  CONTENT  IN  ADS  ELIGIBLE  CASES  WITH 
AND  WITHOUT  ISSUES  DOCUMENTED  IN  JPAS 

The  Adjudication  Decision  Support  (ADS)  research  program  is  developing  rules  for 
automated  screening  of  security  clearance  applications.  Completed  personnel 
security  investigations  with  limited  adverse  content  will  be  approved  by  the  ADS 
system,  while  cases  with  more  substantial  concerns  will  be  automatically  referred 
for  external  adjudication.  In  the  present  study,  preliminary  decision  rules  were 
proposed  and  tested.  This  appendix  illustrates  the  adverse  content  of  cases  that 
were  considered  eligible  for  automated  approval  based  on  the  proposed  rules,  but 
that  had  documented  issues  in  the  Joint  Personnel  Adjudication  System  (JPAS).  A 
small  sample  of  cases  that  were  considered  eligible  and  did  not  have  documented 
issues  in  JPAS  also  are  provided  for  comparison. 

The  adverse  information  contained  in  this  appendix  was  abstracted  from  the 
Defense  Security  Service  Case  Control  Management  System  and  reviewed  by  the 
researchers.  In  addition,  two  personnel  security  experts  were  employed  to  review 
the  abstracted  information  and  to  indicate  for  each  case  whether  or  not  a  potential 
security  concern  was  present.  The  examples  below  represent  cases  in  which  the 
experts  agreed  about  the  presence  or  absence  of  relevant  concerns. 

The  following  examples  provide  a  basis  for  discussing  the  acceptability  of  the 
decision  rules  used  in  this  study,  as  well  as  the  utility  of  issue  categories 
documented  in  JPAS  for  this  type  of  research.  Some  of  the  examples  are  at  the 
boundaries  of  rule  criteria,  or  may  indicate  weaknesses  that  require  revision. 
Conversely,  this  level  of  adverse  content  may  be  acceptable  in  a  risk-management 
framework.  Comprehensive  stakeholder  feedback  will  be  required  as  the  ADS 
research  program  moves  forward. 


B-3 


APPENDIX  B 


EXAMPLES  OF  ADVERSE  CONTENT  FROM  360  ADS  ELIGIBLE 
CASES  WITH  ISSUES  DOCUMENTED  IN  JPAS 

THE  FOLLOWING  EXAMPLES  CLEARLY  CONTAIN  ADVERSE 
INFORMATION  OF  SECURITY  CONCERN  ACCORDING  TO  TWO 
PERSONNEL  SECURITY  EXPERTS  EMPLOYED  FOR  THIS  STUDY 

Case  4 

•  Previous  credit  history:  three  30  day  late,  one  60  day  late,  one  90  day  late 
payments 

•  Business  travel:  20  between  1996  and  2003,  including  China,  United  Arab 
Emirates,  etc.  (not  fully  itemized) 

Case  18 

•  Pending  court  case:  Operating  vehicle  under  the  influence 
Case  22 

•  Father  naturalized  citizen  (1967) 

•  Business  and  pleasure  travel:  18  trips,  including  China,  Hong  Kong,  Japan,  and 
several  European  countries 

Case  24 

•  Previous  credit  history:  one  60  day  past  due 

•  Citation:  Speeding  ticket  (2000) 

•  Arrest:  Disturbing  the  peace,  September  1998,  dismissed  October  1998 
Case  31 

•  Credit  report:  Past  due  balance  $77 

•  Previous  credit  history:  two  30  day,  one  90  day,  two  120  day,  and  one  bad  debt 

•  One  deceased  parent  born  in  Germany 
Case  32 

•  Parents  born  in  Jamaica  (documentation  unclear) 

•  Delayed  entry  program  -  access  suspended 
Case  49 

•  Subject,  parents,  and  siblings  naturalized  citizens  from  the  Philippines  (1988- 
1994).  Spouse  dual  citizen  US/ Philippines.  Subject  is  divorced  from  a  Philippine 
citizen,  alien  registration  number  provided. 

•  Previous  credit  history:  one  30  day  late  payment 
Case  78 
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•  Business  travel:  Hong  Kong  and  Singapore  for  1  and  2  months 

•  Citation:  License  suspended  for  failure  to  appear  (1984),  license  reinstated 
(1984) 

•  Citation:  License  held  in  abeyance  (1985)  pending  driver  improvement  plan  - 
plan  completed  (1985) 

Case  80 

•  Citation:  Inattentive  driving  (2000)  -  $25  fine 

•  Citation:  Following  too  closely  (2001)  -  $25  fine 

•  Citation:  Disregarding  traffic  control  device  (2002)  -  $25  fine 

•  Selective  service:  Registration  number  unrecalled 
Case  87 

•  Arrest:  Underage  drinking  (1999),  misdemeanor,  $67  fine  and  $67  court  costs 
Case  96 

•  Previous  credit  history:  four  30  day  late  payments 

•  Arrest:  Shoplifting  (1998),  misdemeanor  -  dismissed  (2000) 

•  Arrest:  Breaking  and  entering  (1999)  -  dismissed 

•  Citation:  Speeding  and  reckless  driving  (2001)  -  guilty  of  lesser  charge,  $10  fine 
and  $90  court  costs 

Case  97 

•  Father  born  in  Mexico,  no  citizenship  information,  only  knows  date  of  birth 
(Subject  born  in  US) 

Case  99 

•  Citation:  Speeding  (2001)  -  $115  fine 

•  Citation:  Fishing  without  a  license,  pled  guilty,  prayer  for  judgment,  paid  $86 
court  costs 

•  Arrest:  Possession  of  marijuana  (2001),  misdemeanor  -  dismissed 
Case  109 

•  Collection/ charge  off  $271 

•  Previous  credit  history:  two  30  day  late  payments,  one  bad  debt 

•  Nonmortgage  balance:  approximately  $54,000 
Case  no 

•  Previous  credit  history:  two  30,  one  60,  one  90,  one  120  day  late  payment,  and 
one  bad  debt 
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•  Collection/ charge  off  for  $2,012 

THE  FOLLOWING  EXAMPLES  DO  NOT  CLEARLY  CONTAIN  ADVERSE 
INFORMATION  OF  SECURITY  CONCERN  ACCORDING  TO  TWO 
PERSONNEL  SECURITY  EXPERTS  EMPLOYED  FOR  THIS  STUDY 

Case  2 

•  Credit:  Consumer  counseling 

•  Credit  report:  one  30  day  past  due 

•  Nonmortgage  credit  balance  approximately  $51,000 

•  Previous  credit  history:  one  30  day  past  due 
Case  3 

•  Citation:  Failure  to  report  an  accident  in  the  quickest  way  possible  (Class  E 
misdemeanor)  by  waiting  until  the  following  day  -  fine  $100  (2002).  Plead  guilty 
and  paid  $124  (2002). 

Case  5 

•  Counseling:  Marital,  family,  or  grief 
Case  6 

•  Previous  credit  history:  one  60  day  past  due 
Case  8 

•  Credit  report:  two  30  day  late  payments  ($40) 

•  Previous  credit  history:  two  30  day  late  payments 
Case  16 

•  Infraction:  Failure  to  obey  a  traffic  light  (2002)  -  charge  was  nolled  (2002)  and 
records  automatically  erased 

Case  26 

•  Citation:  Speeding  and  improper  passing  (2002)  -  dismissed 

•  Credit  report:  one  30  day  past  due  ($158) 

•  Previous  credit  history:  one  30  day  past  due 

•  Citation:  No  operator’s  license,  charge  pending  with  upcoming  trial 
Case  27 

•  Citizenship  status  unknown  for  one  parent 

•  Pleasure  travel:  3  day  trip  to  Dominican  Republic  (2002) 


B-6 


APPENDIX  B 


Case  33 

•  Citation;  Lesser  charge  of  speeding  (2002)  -  fine  $25,  court  costs  $90 
Case  35 

•  Credit  status;  Debt  consolidation,  $231,000  mortgage,  21  open  credit  accounts 

•  Previous  credit  history;  one  30  day  late  payment 
Case  92 

•  Citation;  Speeding  (2001)  -  fine  $30,  court  costs  $90 

•  Citation;  Speeding  and  failure  to  wear  seatbelt  (2001)  -  fine  $25,  court  costs  $90 

•  Citation;  Failure  to  stop  at  stop  sign  (2000)  -  prayer  for  judgment,  $90  court 
costs 

•  Citation;  Expired  registration  -  fine  $25,  $86  court  costs 

•  Previous  credit  history;  three  30  and  1  60  day  late  payments 
Case  154 

•  Parent  born  in  the  United  Kingdom,  naturalized  (1974),  naturalization  number 
provided 

Case  230 

•  Credit  report;  one  30  day  late  payment  for  $29 

•  Previous  credit  history;  one  30  and  two  60  day  late  payments 

•  Employment  travel;  Ongoing  travel  as  a  flight  attendant  (2001-2003),  including 
Europe,  Canada,  and  Bermuda 

Case  264 

•  Previous  credit  history;  one  30  day  late  payment 
Case  276 

•  Credit  report;  $47  past  due 

•  Credit  status;  22  open  accounts 

•  Previous  credit  history;  eleven  30,  three  60,  one  90,  and  one  120  day  late 
payments 
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EXAMPLES  OF  ADVERSE  CONTENT  FROM  114  ADS-ELIGIBLE 
CASES  WITH  NO  ISSUES  DOCUMENTED  IN  JPAS 

THE  FOLLOWING  EXAMPLES  CLEARLY  CONTAIN  ADVERSE 
INFORMATION  OF  SECURITY  CONCERN  ACCORDING  TO  TWO 
PERSONNEL  SECURITY  EXPERTS  EMPLOYED  FOR  THIS  STUDY 

Case  398 

•  Business  travel:  One  week  trip  to  Israel  (1998) 

Case  405 

•  Credit  report:  $668  past  due 

•  Collections /charge  offs  for  $12,007  related  to  medical  care 
Case  408 

•  Business  travel:  Nonitemized  ongoing  travel  to  South  Korea  -  seemingly  defense 
employment 

•  Previous  credit  history:  one  30  and  one  90  day  past  due  payments 

•  Summary  offense:  Criminal  mischief  for  lighting  pizza  box  and  throwing  into 
dumpster  to  ignite  contents  (1995).  Offense  not  listed  on  security  forms  (2002), 
pled  guilty  -  fine  $100,  court  costs  $72 

Case  414 

•  Attended  school  in  Israel  (1998-2000) 

•  Parent-in-law  born  in  Germany  and  has  dual  citizenship  (U.S. /Israel) 

•  Parent-in-law  born  in  Belgium  (deceased) 

•  Previous  credit  history:  four  30  and  one  60  day  late  payments 
Case  423 

•  Parent-in-law  born  in  Italy,  said  to  be  U.S.  citizen  but  no  documentation 
provided 

•  Spouse-like  relationship  with  naturalized  U.S.  citizen 

•  Counseling:  Marital,  family,  or  grief 

•  Credit  status:  Mortgage  $220,000,  32  open  credit  accounts 

•  Citation:  Speeding  (2002)  -  fine  $1 15.25 
Case  427 

•  Relationship  Otherwise  Known  with  a  Canadian  citizen  living  in  Canada 

•  Foreign  travel:  Pleasure  trip  to  Canada  for  one  month  (Dec.  2002  -  Jan.  2003) 
Case  429 
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•  Spouse  naturalized  citizen  from  the  Philippines 

•  Parents-in-law  Philippine  citizens  (one  deceased) 

Case  442 

•  Father  and  half-brother  were  born  in,  are  citizens  of,  and  live  in  Mexico. 

•  Sibling  is  dual  citizen  U.S./UK,  lives  in  UK 

•  Pleasure  trip  to  Mexico  (2001)  -  1  week 

•  Pleasure  trip  to  UK  (1998)  -  1  week 

•  Credit  status:  24  open  accounts 
Case  447 

•  Business  travel:  Six  trips  to  Germany,  France,  UK  and  Canada  (2000  -  2002), 
each  less  than  1  week 

•  Credit  status:  21  open  accounts 
Case  459 

•  Child  born  abroad  of  U.S.  parents 

•  Parent-in-law  born  in  Hungary,  said  to  be  U.S.  citizen  but  no  documentation 

THE  FOLLOWING  EXAMPLES  DO  NOT  CLEARLY  CONTAIN  ADVERSE 
INFORMATION  OF  SECURITY  CONCERN  ACCORDING  TO  TWO 
PERSONNEL  SECURITY  EXPERTS  EMPLOYED  FOR  THIS  STUDY 

Case  392 

•  Previous  credit  history:  one  30  and  one  60  day  late  payment 
Case  397 

•  Employment  Travel:  Flew  as  a  pilot  to  the  Bahamas  and  Canada  (1999  -  2003) 
Case  403 

•  Both  parents  and  one  sibling  naturalized  citizens  (1985),  one  sibling  naturalized 
(1977) 

Case  410 

•  Born  abroad  of  U.S.  parents,  no  certificate  number  entered 

•  Credit  report:  Past  due  30  days  -  $104 

•  Credit  status:  Nonmortgage  debt  -  $80,000 

•  Credit  report:  Consumer  counseling 

•  Previous  credit  history:  two  30,  two  60,  and  two  90  day  late  payments 
Case  420 
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•  Born  in  German  of  U.S.  parents  (1984),  documented 

•  Pleasure  travel:  Trip  to  Germany  (2000)  -  10  days 
Case  422 

•  Credit  Report:  Debt  consolidation,  consumer  counseling 

•  Previous  credit  history:  five  30  and  one  90  day  late  payments 
Case  426 

•  Business  travel:  Nonitemized  travel  to  the  Bahamas  (1995-1999)  during  military 
service  in  Florida 

•  Credit  status:  21  open  accounts 
Case  435 

•  Credit  status:  21  open  accounts 

•  Citation:  Disobeying  a  traffic  signal  (1998)  -  fine,  fee  unknown 

•  Citation:  Disobeying  a  traffic  signal  (2000)  -  fine,  fee  unknown 

•  Citation:  No  proof  of  insurance  (2002)  -  fine,  fee  unknown 
Case  463 

•  Family  records:  No  address  for  parents  or  sibling  (living) 

•  Credit  report:  two  accounts  30  days  past  due  for  $160 

•  Previous  credit  history:  four  30,  one  60,  and  one  90  day  late  payments 
Case  472 

•  Mother  (deceased)  born  in  Cuba,  said  to  be  U.S.  citizen  but  no  documentation 
provided 

•  Father  naturalized  citizen  from  Cuba,  citizenship  number  provided  but  no  date 

•  Stepmother  naturalized  citizen  from  Costa  Rica,  citizenship  number  provided 
but  no  date 
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LIST  OF  442  KEYWORDS  USED  IN  FINAL  ANALYSIS 
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Table  C-1 

List  of  442  Ke3nvords  Used  In  Final  Analysis 


Keyword 

Usage 

Keyword 

Usage 

Keyword 

Usage 

$ 

FAC 

ary  an 

WAC 

chapter  13 

WAC 

absentee 

FAC 

assailant 

WAC 

chapter  7 

WAC 

abus 

FAC 

assault 

WAC 

charge 

WAC 

accident 

FAC 

asylum 

WAC 

charges  dropped 

WAC 

accomplice 

WAC 

attorney 

WAC 

chat  room 

FAC 

account 

FAC 

authenticat 

FAC 

child  custody 

WAC 

accus 

FAC 

avenge 

FAC 

child  endangerment 

WAC 

addict 

FAC 

B&E 

WAC 

Child  enforcement 

WAC 

adjournment 

WAC 

bad 

WAC 

child  support 

WAC 

adjudicat 

FAC 

bail 

WAC 

chronic 

WAC 

advers 

FAC 

balance 

WAC 

civil  case 

WAC 

advis 

WAC 

bankrup 

FAC 

civil  conviction 

WAC 

affluence 

WAC 

barbituate 

WAC 

claim 

WAC 

aggravat 

FAC 

battery 

WAC 

cocaine 

WAC 

alcohol 

FAC 

behavior 

WAC 

collection 

WAC 

alcohol  awareness 

WAC 

bench 

WAC 

community  service 

WAC 

alcohol  problem 

WAC 

bey  at 

WAC 

computer  download 

WAC 

alcoholics  anonymous 

WAC 

bigamy 

WAC 

computer  virus 

WAC 

alien 

WAC 

bigot 

WAC 

conceal 

FAC 

allegation 

FAC 

bipolar 

WAC 

condition 

FAC 

al-q 

FAC 

blood 

WAC 

confederate 

WAC 

altercation 

WAC 

blow  up 

WAC 

conflict 

WAC 

amount 

WAC 

bomb 

WAC 

confront 

FAC 

amphetamine 

FAC 

bouncer 

WAC 

contempt  of  court 

WAC 

anarch 

FAC 

brag 

WAC 

convict 

FAC 

anger 

WAC 

brainwash 

WAC 

counseling 

WAC 

Anger  Management  Class 

WAC 

breach 

WAC 

count 

WAC 

anxiety 

FAC 

breath 

FAC 

county  court 

WAC 

anxiety  attack 

WAC 

brib 

FAC 

court 

WAC 

apathy 

WAC 

bruise 

WAC 

court  date 

WAC 

apology 

WAC 

burglary 

WAC 

court  enforcement 

WAC 

appeal 

WAC 

burn 

WAC 

court  ordered 

WAC 

arab 

FAC 

buttocks 

WAC 

CPU21 

WAC 

arbitral 

FAC 

cannibis 

WAC 

credit 

FAC 

arbitration 

WAC 

careless 

WAC 

credit  card 

WAC 

argu 

FAC 

carnal 

WAC 

crim 

FAC 

argument 

WAC 

case 

WAC 

cruelty 

WAC 

armageddon 

WAC 

caught 

WAC 

cultural  attache 

WAC 

arraign 

FAC 

CDS 

WAC 

curfew 

WAC 

arrearage 

WAC 

certificate  of  disposition 

WAC 

cyber 

WAC 

arrest 

FAC 

certificate  of  relief 

WAC 

damage 

FAC 

arson 

WAC 

chapter 

WAC 

date  incurred 

WAC 

Article  15 

WAC 

chapter  1 1 

WAC 

david  koresh 

WAC 
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Keyword 

Usage 

Keyword 

Usage 

Keyword 

Usage 

debt 

FAC 

ethic 

FAC 

homocide 

WAC 

debt  collection 

WAC 

expatriat 

FAC 

hurt 

FAC 

debt  satisfied 

WAC 

experiment 

FAC 

illegal 

WAC 

debtor  discharge 

WAC 

expir 

FAC 

immigrat 

FAC 

defense  attache's  office 

WAC 

explosive 

FAC 

Impair 

FAC 

defense  force 

WAC 

export 

FAC 

imprison 

FAC 

defensive  driver 

FAC 

expulsion 

WAC 

incarcerat 

FAC 

delinq 

FAC 

expunge 

FAC 

incest 

WAC 

deni 

FAC 

extort 

FAC 

incident 

WAC 

deny 

WAC 

extramar 

FAC 

infidel 

FAC 

depress 

FAC 

fail 

FAC 

insomnia 

FAC 

derelict 

FAC 

fals 

FAC 

insubordina 

FAC 

derog 

FAC 

fanatic 

WAC 

internal 

WAC 

desertion 

WAC 

fbi 

WAC 

interven 

WAC 

diplomat 

FAC 

fee 

FAC 

intoxicat 

FAC 

disagree 

FAC 

felony 

WAC 

irate 

WAC 

disaster  management 

WAC 

final  disposition 

WAC 

irreconcil 

FAC 

discharge 

FAC 

financ 

FAC 

IRS 

WAC 

dishonor 

FAC 

fine 

FAC 

islam 

FAC 

disloyal 

FAC 

fire 

FAC 

iail 

FAC 

dismiss 

FAC 

firearm 

FAC 

ieopardy 

WAC 

disorder 

WAC 

FMLN 

WAC 

iew 

FAC 

dispos 

FAC 

fond 

FAC 

jihad 

WAC 

distrust 

FAC 

foolish 

FAC 

joint  custody 

WAC 

disturb 

FAC 

foreclos 

FAC 

judg 

FAC 

divorce  settle 

FAC 

foreign  business 

WAC 

juvenile 

WAC 

dmv 

WAC 

forge 

FAC 

ketamine 

WAC 

docket 

WAC 

FOUO 

WAC 

kill 

WAC 

doctor 

WAC 

fugutive 

WAC 

KKK 

WAC 

domestic 

WAC 

gambl 

FAC 

knife 

WAC 

donat 

FAC 

gang 

WAC 

LAPD 

WAC 

drink 

FAC 

garnish 

FAC 

larceny 

WAC 

dropped 

WAC 

gram 

WAC 

late 

WAC 

drug 

FAC 

guilt 

FAC 

licens 

FAC 

drunk 

FAC 

gun 

WAC 

lied 

WAC 

dual 

WAC 

had  it  coming 

WAC 

lien 

WAC 

dui 

WAC 

halcion 

WAC 

liquor 

WAC 

dwi 

WAC 

Hamas 

WAC 

lithium 

WAC 

ecstacy 

WAC 

hangover 

WAC 

loan 

FAC 

embezzl 

FAC 

harass 

FAC 

machette 

WAC 

emigrat 

FAC 

health  care  provider 

WAC 

madd 

WAC 

emotion 

WAC 

heathen 

FAC 

mafia 

WAC 

episode 

WAC 

Hezb 

FAC 

magistrate 

WAC 

espionage 

WAC 

hit 

WAC 

manifesto 

WAC 

estate 

WAC 

hitler 

WAC 

marijuana 

WAC 

estimated  value 

WAC 

home  country 

WAC 

martial 

WAC 
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Keyword 

Usage 

Keyword 

Usage 

Keyword 

Usage 

mecca 

WAC 

pay 

FAC 

reposses 

FAC 

mental 

WAC 

percocet 

WAC 

reprimand 

FAC 

mercenar 

FAC 

personality  conflict 

WAC 

resignation 

WAC 

meth 

FAC 

picked-up 

WAC 

restrict 

FAC 

methamphetamine 

FAC 

pilgrim 

FAC 

retaliat 

FAC 

militan 

FAC 

pimp 

FAC 

revok 

FAC 

minor 

WAC 

plaintiff 

WAC 

revolver 

WAC 

MIP 

WAC 

polic 

FAC 

riot 

FAC 

misappropriate 

FMC 

polygamy 

WAC 

ritalin 

WAC 

misconduct 

WAC 

porn 

FAC 

Rommel 

WAC 

molest 

FAC 

porn  site 

FAC 

sabotage 

FAC 

money 

WAC 

positive 

WAC 

satis 

FAC 

mortgage 

FAC 

possess 

FAC 

satop 

WAC 

mosque 

WAC 

precinct 

FAC 

SCIF 

WAC 

motion 

FAC 

prescrib 

FAC 

SDS 

WAC 

motor 

WAC 

prescription 

WAC 

seat  belt 

WAC 

mph 

WAC 

prison 

FAC 

seatbelt 

FAC 

MPS 

WAC 

probat 

FAC 

second  amendment 

WAC 

MSA 

WAC 

promis 

FAC 

sedition 

WAC 

MTA 

WAC 

proof 

WAC 

segregat 

FAC 

murder 

FAC 

property 

WAC 

Semitic 

WAC 

mushroom 

FAC 

proprietor 

WAC 

Serzone 

WAC 

mu  slim 

WAC 

prostitut 

FAC 

settle 

FAC 

n.j.s 

WAC 

protest 

FAC 

sex 

FAC 

narcotic 

FAC 

prozac 

WAC 

sheriff 

FAC 

nasdaq 

WAC 

psych 

FAC 

shoot 

FAC 

nazi 

FAC 

punch 

FAC 

shoplift 

FAC 

new  world  order 

WAC 

punish 

FAC 

side  effects 

WAC 

nigger 

FAC 

pur^e 

FAC 

situation 

WAC 

NLETS 

WAC 

pursuit 

WAC 

skinhead 

WAC 

no  contest 

WAC 

racial 

WAC 

small  claims 

WAC 

nolo 

WAC 

rap  sheet 

WAC 

sodomy 

WAC 

non-US  citizen 

WAC 

rape 

FAC 

speed 

FAC 

not  a  citizen 

WAC 

real  estate 

WAC 

speeding  ticket 

WAC 

null  process 

WAC 

reckless 

WAC 

ssn 

WAC 

nystatin 

WAC 

recognizance 

WAC 

stalk 

FAC 

obscene 

WAC 

red  badge 

WAC 

steal 

FAC 

obsess 

FAC 

red  light 

WAC 

stole 

FAC 

offens 

FAC 

reduc 

FAC 

stress 

FAC 

ounce 

WAC 

re-entry 

WAC 

substance 

WAC 

oxycodone 

WAC 

refugee 

WAC 

subversive 

WAC 

paid 

WAC 

regulat 

FAC 

suit 

WAC 

palimony 

WAC 

rehab 

FAC 

supremacy 

WAC 

pander 

FAC 

rehir 

FAC 

suspect 

FAC 

parol 

FAC 

remission 

WAC 

suspicion 

WAC 

paxil 

WAC 

renounc 

FAC 

swastika 

FAC 
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Keyword 

Usage 

Keyword 

Usage 

Keyword 

Usage 

symptom 

FAC 

turner  diaries 

WAC 

violen 

FAC 

tax 

FAC 

u.s.c 

WAC 

visa 

FAC 

the  n  word 

WAC 

uif 

WAC 

waiver 

WAC 

theft 

WAC 

ultra  conservative 

WAC 

warn 

FAC 

therap 

FAC 

undesir 

FAC 

warrant 

FAC 

threat 

FAC 

unfavorable 

WAC 

wellbutrin 

WAC 

ticket 

FAC 

unlisted 

WAC 

witness 

FAC 

time  share 

WAC 

urinalysis 

WAC 

worthless 

WAC 

treason 

WAC 

USBC 

WAC 

worthless  check 

WAC 

treasury 

WAC 

vagran 

FAC 

zoloft 

WAC 

trial 

WAC 

vehicle 

WAC 

Zyprexa 

WAC 

tried  mar 

FAC 

verdict 

WAC 

troubl 

FAC 

violat 

FAC 

Note.  FAC  =  Fragment  Any  Case,  WAC  =  Word  Any  Case. 
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